Document of The World Bank Report No.: 82308-TN PROJECT PERFORMANCE ASSESSMENT REPORT TUNISIA NATURAL RESOURCES MANAGEMENT PROJECT (LOAN NO. 4162; TF025830) NORTHWEST MOUNTAINOUS AND FORESTRY AREAS DEVELOPMENT PROJECT (LOAN NO. 7151,TF026699) November 6, 2013 IEG Public Sector Evaluation Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Tunisian Dinar (TD) 1997 US$1.00 TD1.1059 1998 US$1.00 TD 1.1387 1999 US$1.00 TD 1.1862 2000 US$1.00 TD 1.3707 2001 US$1.00 TD 1.4387 2002 US$1.00 TD 1.4217 2003 US$1.00 TD 1.2885 2004 US$1.00 TD 1.2455 2005 US$1.00 TD 1.2974 2006 US$1.00 TD 1.3310 2007 US$1.00 TD 1.2814 2008 US$1.00 TD 1.2321 2009 US$1.00 TD 1.3503 Abbreviations and Acronyms CRDA Regional Agricultural Development Offices DCs Development Committees DGFIOP General Directorate of Finance, Investments and Professional Organizations GDA Agricultural and Fisheries Development Associations ICR Implementation Completion and Report IEG Independent Evaluation Group IEGPS IEG Public Sector Evaluation IPA Integrated Participatory Approach IGAs Income-Generating Activities MARH Ministry of Agriculture and Water Resources NRMP Natural Resources Management Project NWMFP Northwest Mountainous and Forestry Areas Development Project ODESYPANO The Northwest Forestry and Pastoral Development Agency PIU Project Implementation Unit PPAR Project Performance Assessment Report Fiscal Year Government: January 1 – December 31 Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Public Sector Evaluation : Mr. Emmanuel Jimenez Manager, IEG Public Sector Evaluation : Ms. Marie Gaarder Task Manager : Ms. April Connelly iii Contents Principal Ratings ............................................................................................................... vii Key Staff Responsible....................................................................................................... vii Preface................................................................................................................................ ix Summary ............................................................................................................................ xi 1. Background and Context................................................................................................. 1 2. Natural Resource Management Project (1997–2004) ..................................................... 3 Background and Project Context .................................................................................... 3 Objectives, Design, and their Relevance ........................................................................ 3 Objectives ................................................................................................................... 3 Relevance of Objectives ............................................................................................. 4 Design ......................................................................................................................... 4 Implementation Arrangements .................................................................................... 6 Monitoring and Evaluation Design ............................................................................ 6 Relevance of Design ................................................................................................... 7 Implementation ............................................................................................................... 7 Planned vs. Actual Expenditure by Component ......................................................... 7 Implementation Experience ........................................................................................ 8 Implementation of Monitoring and Evaluation......................................................... 10 Safeguards Compliance ............................................................................................ 11 Achievement of the Objectives ..................................................................................... 11 Objective 1: Sustainable Natural Resources Management, in Particular of Crop and Range Land in Severely Degraded Zones ................................................................. 11 Outputs ...................................................................................................................... 11 Outcomes................................................................................................................... 11 Objective 2: Improving Agricultural Productivity .................................................... 12 Outputs ...................................................................................................................... 12 Outcomes................................................................................................................... 13 Objective 3: greater involvement of resource users in development programs ........ 14 Outputs ...................................................................................................................... 14 Outcomes................................................................................................................... 15 Efficiency ...................................................................................................................... 16 This report was prepared by April Connelly, with input from Camille Konate and Maamri Akremi who also provided support for the IEG mission to Tunisia in October 2012. The report was peer reviewed by Grant Milne and panel reviewed by Christopher D. Gerrard. Marie Charles provided administrative support. iv Ratings .......................................................................................................................... 16 Outcome .................................................................................................................... 16 Risk to Development Outcome ................................................................................. 17 Bank Performance ..................................................................................................... 18 Borrower Performance .............................................................................................. 18 Monitoring and Evaluation ....................................................................................... 19 3. Northwest Mountainous and Forestry Areas Development Project (2002–2009) ........ 20 Background and Project Context .................................................................................. 20 Objectives, Design, and their Relevance ...................................................................... 21 Objectives ................................................................................................................. 21 Relevance of Objectives ........................................................................................... 21 Design ....................................................................................................................... 22 Geographic Scope of the Operation ......................................................................... 22 Implementation Arrangements .................................................................................. 23 Monitoring and Evaluation Design .......................................................................... 24 Relevance of Design ................................................................................................. 24 Implementation ............................................................................................................. 24 Planned versus Actual Disbursements ...................................................................... 24 Implementation Experience ...................................................................................... 26 Monitoring and Evaluation Implementation and Use ............................................... 27 Safeguards Compliance ............................................................................................ 27 Achievement of the Objectives ..................................................................................... 29 Objective 1: Enhance Household Incomes And Socio-Economic Conditions Through The Improvement And Diversification Of The Agricultural And Pastoral Production Systems And The Promotion Of Off-Farm Income-Generating Activities ................................................................................................................................... 29 Outputs ...................................................................................................................... 29 Outcomes................................................................................................................... 31 Objective 2: Ensure Sustainable Management of the Natural Resources Through the Improvement and Diversification of the Agricultural and Pastoral Production Systems and the Promotion of Off-Farm Income-Generating Activities ................. 31 Outputs ...................................................................................................................... 32 Outcomes................................................................................................................... 32 Efficiency ...................................................................................................................... 32 Ratings .......................................................................................................................... 33 Outcome .................................................................................................................... 33 v Risk to Development Outcome ................................................................................. 33 Bank Performance ..................................................................................................... 35 Borrower Performance .............................................................................................. 37 Monitoring and Evaluation ....................................................................................... 37 4. Lessons .......................................................................................................................... 38 References ......................................................................................................................... 39 Annex A. Basic Data Sheet ............................................................................................... 41 Annex B. List of Persons Met ........................................................................................... 46 Boxes Box 1. The Integrated Participatory Approach to Local Development Planning ............... 2 Tables Table 1. Project Costs by Component (in US$ million equivalent) ................................... 9 Table 2. Sustainable Natural Resources Management Achievements .............................. 12 Table 4. Rural Infrastructure Achievements ..................................................................... 13 Table 5. Agriculture Productivity Achievements ............................................................. 13 Table 6. Sustainable Natural Resources Management Achievements .............................. 15 Table 7. Project Cost by Component (in USD million equivalents)................................. 26 Table 8. Agriculture Production System Results .............................................................. 30 vii Principal Ratings Natural Resources Management Project ICR* ICR Review* PPAR Outcome Moderately Satisfactory Satisfactory Unsatisfactory Risk to Substantial/Likely Substantial/Likely Development Significant (sustainability) (sustainability) Outcome Bank Performance Satisfactory Satisfactory Moderately Satisfactory Borrower Satisfactory Satisfactory Moderately Satisfactory Performance * The Implementation Completion Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEGWB product that seeks to independently verify the findings of the ICR. Northwest Mountainous and Forestry Areas Development Project ICR* ICR Review* PPAR Outcome Satisfactory Moderately Satisfactory Moderately Unsatisfactory Risk to Development Moderate Significant Moderate Outcome Bank Performance Satisfactory Moderately Satisfactory Moderately Unsatisfactory Borrower Satisfactory Moderately Satisfactory Moderately Satisfactory Performance * The Implementation Completion Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEGWB product that seeks to independently verify the findings of the ICR. Key Staff Responsible Natural Resources Management Project Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal L. Msellati M. Wilson D. Ritchie Completion Douglas W. Lister Letitia A. Obeng Theodore O. Ahlers Northwest Mountainous and Forestry Areas Development Project Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Idah Z. Pswarayi- Petros Aklilu Pedro Alba Riddihough Completion Lucie Tran Huong- Luis Constantino Françoise Clottes Giang/ Garry Charlier viii IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. ix Preface The Natural Resources Management Project (Loan No. 4162; TF-25830) was approved on May 13, 1997 and became effective on October 15, 1997. It was financed by an IBRD credit of US$26.5 million equivalent. At project closure US$20.9 million equivalent had been disbursed. US$0.95 million equivalent of the original loan amount was cancelled. Project costs were almost one-third less than expected, primarily because, in US dollar terms, the Tunisian Dinar depreciated by 27 percent during implementation. Beneficiaries contributed slightly more than planned (14 percent of total project cost compared to the 13 percent forecast at appraisal). The project closed on June 30, 2004, one year after the original closing date of June 30, 2003, mainly owing to a lag in implementation of a European Union-financed component for staff training. The Northwest Mountainous and Forestry Areas Development Project (Loan No. 7151 and TF-26699) was approved on October 31, 2002 and became effective on July 14, 2003. It was financed by an IBRD loan of US$34 million equivalent, of which 24.5 percent of the original loan amount in Euros was cancelled due to a substantial appreciation of the Euro. At project closure US$34.48 million equivalent had been disbursed. Total project costs were US$44.74 million equivalent, slightly less than the US$44.86 million appraisal estimate. The project closed on July 31, 2009, seven months after the original closing date of December 31, 2008. The closing date was extended to complete cropping activities that overlapped two calendar years. This report is based on a review of project documents, including the Implementation Completion and Results Report, the Project Appraisal Document, legal documents and project files, and on discussions held with Bank staff involved in the project. It is also based on an IEG assessment mission to Tunisia that was conducted in October 2012. IEG held meetings in Tunis and conducted site visits in the departments of Beja, Jendouba, and Medenine to interview regional staff and local communities. The mission expresses its appreciation for the generous time and attention given by the Borrower and all concerned parties. A list of persons met by the mission is in Annex B. The assessment verifies the results of each operation and assesses their sustainability in the aftermath of the Tunisian revolution. This evaluation will also serve as an input into IEG‘s Country Program Evaluation of Tunisia. Following standard IEG procedures, copies of the draft report has been sent to government officials and agencies for their review and comments. No comments have been received. xi Summary This Project Performance Assessment Report assesses the development effectiveness of two Tunisia participatory natural resource management projects: the Natural Resources Management Project (1997–2004) and the Northwest Mountains and Forest Areas Project (2002–2009). Both projects were concerned with addressing land degradation and enhancing the local population’s socio-economic conditions. Both projects had similar content, combining construction of soil and water conservation works, financing for agriculture and non-farm income-generating activities, and the construction of basic rural infrastructure. Both projects employed an integrated participatory approach that facilitated greater engagement of local communities in the local development planning process through support for the preparation of community development plans. The projects differed with respect to the type of implementation agency in charge of project coordination and the areas of the country in which they were carried out. The Natural Resources Management Project was a pilot to test the feasibility of implementing the integrated participatory approach within the Ministry of Agriculture’s Regional Agriculture Development Offices as a first step to mainstreaming the approach throughout the country. The project was carried out in highly degraded areas in the country’s three main agro-ecological zones. The Northwest Mountains and Forest Areas project was the third in a series of World Bank financed projects implemented by the Northwest Forestry and Pastoral Development Agency, a semiautonomous government agency created to ensure the management of natural resources to minimize soil erosion and silting of dams in mountainous areas in the northwest zone of the country. The northwest region is of high strategic importance to the country since its watersheds provide 75 percent of the national water supply. It is also one of the most isolated areas in the country and its population has suffered some of the lowest socioeconomic conditions in Tunisia. The project aimed to further strengthen the participatory approach and expand its use. Natural Resources Management Project (1997–2004) The objectives of the Natural Resources Management project were to assist the Borrower in sustaining natural resource management, in particular on crop and range land, and improving productivity attained with greater involvement of resource users in development programs. The project met most of its targets for the construction of soil and water conservation works. These works aimed to stabilize and reverse erosion, restore soil fertility and contribute to water conservation but the project did not measure actual results such as changes in soil erosion, soil fertility, sedimentation or aquifer recharge levels. Targets for the establishment of pastoral plantations were not met and reported agricultural productivity gains were below expectations due to four consecutive years of drought that occurred during project implementation. Support for small-scale irrigation works and extension efforts on water conservation techniques are reported to have helped to expand agricultural areas and reduce farmers’ workloads. But no evidence was provided to substantiate this claim. Output targets related to involving resource users in development programs through the implementation of the participatory approach were met or exceeded. The project approach facilitated the development of partnerships between local communities and government line agencies. The xii number of Participatory Development Plans prepared by local development committees reached appraisal estimates but the number of households covered by these plans (12,800 households) exceeded appraisal estimates (7,000 households) due to higher than anticipated population density in participating communities. The number of annual contracts approved to finance activities prioritized in the participatory development plans exceeded appraisal forecasts and the beneficiary share of the cost of soil and water conservation was in line with appraisal forecasts. However, a durable mechanism to engage resource users in development programs has not yet been established. The community-level development committees that served as the interface between the communities and the local development process were project-driven entities, most do not have formal legal status and many ceased to operate following project closure. The overall outcome of the natural resources project is rated Moderately Unsatisfactory. The project objectives were substantially relevant to Bank strategies and Government policies in effect when the project was approved and remain relevant to current strategies. Relevance of design was modest. The project design included a number of activities relevant to achieving its objectives, combining protection measures with productions measures to spark community interests. But the design lacked measures to address marketing and input procurement that are key constraints to link agricultural productivity to income generation. Achievement of the three project objectives was modest as was efficiency. The risk to development outcome is Significant. Gains made by project closure were at an incipient stage and in need of consolidation. Although a follow-on project was developed, there was a 6 year delay between closure of the project and initiation of the follow-on phase. In the interim many of the capacity building gains in the local community and the Regional Agricultural Centers have eroded. The community development committees lacked formal legal status and most ceased to operate following the project. Many of the local contractors hired who carried out work prioritized through annual contracts were unable to find employment following the project. Further disruptions to consolidating project gains occurred after the 2011 revolution. Many of the formally recognized producer organizations have broken down since communities viewed them as arms of the administration and tensions have risen between communities and the Regional Agriculture Development Centers and other government entities. Bank performance is rated Moderately Satisfactory. Project preparation was based on analytical studies and consultations with targeted communities. But some risks were not adequately identified and a formal framework to guide the engagement with co-financing and implementation partners was not put in place resulting in ad-hoc interactions between the various parties. M&E was also inadequate. Supervision missions were frequent and teams appropriately intervened to limit the number of Participatory Development Plans prepared in each year to improve the quality of the plans prepared and mitigate the threat of depleting the projects budget before project completion. Borrower performance is rated Moderately Satisfactory. The government demonstrated commitment to the project through its active engagement in project preparation and financing commitments, but could have played a more active role in trying to resolve the delay in the availability of the EU grant. The central project implementation unit played a proactive role in project coordination and took initiative in addressing implementation challenges, implementing a temporary training program at the xiii beginning of the project to compensate for delays in the EU-financed training component. But the agency failed to replace project staff that left their positions during implementation and shares responsibility with the Bank for the lack of a sound M&E system. Northwest Mountains and Forest Areas Project (2002–2009) The main objectives of the project were to increase household incomes and enhance the socio-economic conditions of the population in the Project Area, while ensuring sustainable management of the natural resources, through the improvement and diversification of the agricultural and pastoral production systems and the promotion of off-farm income- generating activities. The project employed a participatory approach to achieve its objectives. The project succeeded in meeting many of its output targets for agriculture and livestock production activities, rural infrastructure and soil and water conservation investments, but the project fell short in measuring results from these activities. Progress was made in terms of increased yields, diversification of agricultural production systems, and land use rationalization. A land consolidation pilot was implemented and had many positive results including a reduction in the fragmentation of plots and increase in the average size of farm plots which in turn reduced production costs (mechanization, transport). Conflicts over land rights were reduced in the areas where consolidation was successful, while in some areas consolidation efforts were stalled as a result of conflicts that could not be resolved. Less success was achieved from efforts to diversify income sources. Pilot activities to develop alternative income-generating projects were constrained by difficulties in accessing credit. Implementation of a pilot to promote greater engagement of forest communities in forest management activities was delayed and no outcomes were measured. In terms of income and socio-economic enhancement outcomes, the project completion report noted that average household income in the region as a whole increased from TD 2,050 in 2003 to TD 3,784 in 2009 in constant terms but no control groups were established to assess the extent that this is attributable to project activities. Unemployment rates for the region as a whole also declined from 19 percent in 2003 to 16 percent in 2009 but the same attribution issues apply. Access to roads increased from 50 percent to 81 percent and access to potable water increased from 69 percent to 81 percent over the project time period. In terms of natural resources management goals, soil and water conservation works, rangeland management and agro- forestry systems were carried out on 54,880 hectares exceeding the target of 48,000 ha. While vegetation and forest cover in the project region increased from 32 percent to 38 percent, there is insufficient information to assess the extent to which this change can be attributed to project activities. Actual changes in soil loss, siltation of dams, and water recharge were not assessed and there is no information on changes in communities’ natural resources management practices or the incentives that they have to maintain the project’s investments in soil and water conservation works, rangeland improvements or agro-forestry. The overall outcome of the Northwest Mountains and Forest Areas Project is rated Moderately Unsatisfactory. The project objectives were substantially relevant to Bank strategies and Government policies in effect when the project was approved and remain relevant to current strategies. Relevance of design was also substantial. Achievement of the objectives was modest due to lack of outcome-level evidence and attribution questions. Efficiency was also modest. The risk to development outcome is Moderate. A follow-on xiv project is ongoing, though it has experienced a number of start-up delays related to implementation challenges that emerged in the aftermath of the revolution. The greatest risk is the weakness of community organizations. The continued presence of ODESYPANO has been a positive factor in terms of ensuring continuity of community planning despite a leadership crisis affecting many community organizations following the revolution. Nonetheless, most community organizations are project-driven entities and there is little evidence of enhanced community capacity to indicate that they are capable of operating effectively without the project’s support. Towards the end of the project the government decided to make the agency a permanent entity but there are questions about the agency’s sustainability in the absence of World Bank financing. Since the agency’s inception its budget has been financed jointly by the government and World Bank project funds. The Bank performance is rated Moderately Unsatisfactory. Project preparation was based on solid analysis and improved upon the design of prior projects with the introduction of some innovations, but there were weaknesses in the M&E indicators and some risks were overlooked. Supervision missions provided adequate technical and fiduciary backstopping but the supervision missions were not as frequent as they should have been and the teams lacked expertise in safeguards and monitoring and evaluation. Performance was also weakened by inadequate supervision of safeguards. Borrower performance is rated Moderately Satisfactory. The government showed a high degree of commitment and ownership toward the project design, preparation and implementation. Counterpart funds were made available on time and more than originally planned. The implementing agency performed effectively in design and implementation but there were shortcomings in monitoring and evaluation and the supervision of environmental and social safeguards. Lessons The experience of the two projects yields the following lessons:  Weaknesses in the structures that are used to integrate communities into the development process can undermine securing genuine collaboration of communities over the long term. The current forms of organization available to communities in Tunisia have a number of limitations that hamper their utility as a vehicle to formally represent the interests of the community as a whole. It is important that community organizations have sufficient legal standing to ensure their legitimacy in representing the communities’ priorities to government agencies, that communities view the leaders of such organizations as legitimate, and that all segments of the population are represented.  Demonstrating the effectiveness of natural resource management interventions requires attention to monitoring and evaluation of actual outcomes. Despite three decades of World Bank support to integrated rural development projects with significant investments in soil and water conservation works, no assessment has been made to determine the actual impacts of these interventions on erosion, soil fertility, groundwater recharge, or dam siltation. To date reporting of natural resources management achievements has been based on assumed impacts. Measuring actual results is not only important for understanding whether the intended development impact has been achieved, but also for assessing the cost- xv effectiveness of the measures supported and generating information on what needs to be improved.  Effective promotion of income-generating projects requires more than support for production. Attention to markets and ensuring the availability of credit is also crucial. Under both projects support for alternative income-generating projects was limited to training and technical support for production side issues. The projects had limited success in establishing successful income-generating enterprises. Many of the training participants were unable to access existing sources of credit. Income-generating activities were also hampered by a lack of market studies. Richard G. Scobey Acting Director-General Evaluation 1 1. Background and Context 1.1 Soil erosion is one of Tunisia’s most significant natural resources management concerns. It contributes to a decline in the productivity of agricultural land, a primary source of income for the rural population, and threatens the sustainability of dams.1 It is estimated that about half of the country’s arable land is severely eroded and 13,000 to 23,000 hectares of topsoil are lost each year (World Bank 2010). The rate of dam siltation is estimated at 0.8 percent per year (UNCDS 2012). 1.2 Over the past few decades the government has invested heavily in soil and water conservation measures but with limited results. Since 1970 public investments in soil and water conservation have averaged US$35 million annually and cover over 1 million hectares (out of 4.8 million hectares of arable land). These investments were implemented through a top-down approach that involved the construction of large scale soil and water infrastructure and the establishment and expansion of protected areas. This approach resulted in poor sustainability due to weak ownership by the communities ultimately in charge of local land and water management. 1.3 In an attempt to improve results the Government decided to reorient its approach from a top-down model to a more collaborative approach of engaging with resource users (World Bank 2002). It was believed that direct involvement of populations would result in a better formulation and prioritization of development activities and ensure greater sustainability (World Bank 2008). The shift to a more participatory approach occurred in the course of implementing two prior projects financed by the World Bank in the Northwest Region of the country and resulted in the development of a participatory planning model known as the Integrated Participatory Approach. The key features of this model are presented in Box 1. 1.4 The Integrated Participatory Approach is currently used by all rural development projects in the country. But the approach has not yet been institutionalized through a permanent government agency or scaled up nationwide. The World Bank continues to support the evolution of the IPA model in general and through two ongoing operations. 1.5 This review assesses the development effectiveness of two of the World Bank financed projects that have employed the integrated participatory approach. 1 Precipitation is irregular in the north and some parts of central regions of the country, with periods of long droughts followed by short but intense rainfall that results in heavy erosion and flooding. Runoff prevents rainwater infiltration in to the soil, leads to insufficient groundwater replenishment and increased sedimentation. Wind erosion threatens land productivity in the more arid regions in the Central and South of the country. 2 Box 1. The Integrated Participatory Approach to Local Development Planning What is the Integrated Participatory Approach? The integrated participatory approach (IPA) is a mechanism to involve communities directly into the local development planning process. The IPA establishes partnerships between communities and public institutions at the regional and sub-regional levels. The IPA is believed to produce more relevant and sustainable results compared to a traditional top- down approach. It is assumed that direct involvement of populations will result in better formulation and prioritization of development activities and ensure greater durability. Who is involved? Implementation of the approach involves three main sets of actors:  A facilitating entity that mobilizes the community, facilitates the preparation of development plans and establishes partnerships with other (government and non government) development actors. (To date this role has been filled by the implanting unit of rural development projects that have implemented the approach)  Local communities organized by common socio-territorial unit.  Local government administration, regional representation of relevant line agencies, and other (government and nongovernment) actors involved with local and regional development. How does it work? Communities are organized according to a common socio-territorial unit (village or cluster of villages) and elect a committee that serves as the main interface with the project and public officials. A participatory development plan (PDP) is prepared that reflects the communities’ development priorities over the 5 years duration of the plan. Activities in the PDP are funded through annual contracts (CPA) that are signed with the project for implementation. Both PDP and CPA are approved by local government administrative bodies, the Regional Council and Local Council. In the course of preparation the technical feasibility of proposed activities are assessed by a relevant government agency. Who funds implementation of the activities in the development plans? The activities prioritized in the PDPs are funded by the budgets of the development project that is implementing the approach, relevant government agencies, and other development partners. Typically the project budget only finances activities in PDPs/CPAs that are related to agricultural and pastoral production, income-generating activities, consolidation, protection and sustainable management of natural resources, and improvement of basic rural infrastructure (primarily rural roads and water supply). Other activities prioritized in these plans are funded directly by relevant sector ministries, governorates and delegations, or main partner agencies, from their own budgets. Communities are also required to contribute to funding of the activities implemented through both in kind and financial contributions. How does the IPA differ from other participatory projects implemented globally? Compared to many community driven development projects globally the extent of community participation is limited. It is more of a negotiation tool for programming technical development projects at the local level as opposed to a program that is fully driven by the community. The model allows communities to drive the development agenda in terms of expressing their priorities and inserting them into local development plans. But their role in implementation is limited and no money is transferred directly to the community, rather the project administers the funds on their behalf. The model is continuing to evolve towards a more demand-driven approach. Source: World Bank 1997, 2002, 2010b, IEG interviews 3 2. Natural Resource Management Project (1997–2004) Background and Project Context 2.1 The project was prepared to test the feasibility of mainstreaming and scaling up the Integrated Participatory Approach (IPA) to local development planning that had been developed by the Northwest Forestry and Pastoral Development Agency (ODESYPANO). ODESYPANO is a state-owned semi-autonomous agency with a mandate for protecting vulnerable ecosystems and developing rural infrastructure in the mountainous areas of the Northwest region of Tunisia. ODESYPANO started with a top-down approach but progressively adopted a participatory and integrated approach involving local communities in decision making and execution of projects. ODESYPANO found that the use of this participatory approach generated greater acceptance for soil and water conservation interventions on private and community lands. Investments implemented through the IPA were also believed to be more sustainable. 2.2 The Project aimed to build the capacity of the Ministry of Agriculture for implementing the integrated participatory approach through its regional offices, the Regional Commissariats for Agricultural Development (CRDA). Mainstreaming the IPA within the CRDAs was intended to enhance the institutional sustainability of the model, as ODESYPANO’s status was originally meant to be temporary. The management costs of implementing the model through the CRDA were also expected to be lower. 2.3 The project was designed as a pilot. Staff in the CRDA did not have experience with participatory methods so the pilot aimed to strengthen the CRDA staff’s capability in participatory management through training and recruitment. The CRDA was also organized to take a sectoral approach to development and would need to transition to a multi- disciplinary approach. Based on the pilot results the government would consider scaling up through other CRDAs in the rest of the country. Objectives, Design, and their Relevance OBJECTIVES 2.4 The objectives of the project as set out in the lending agreement (p. 13) are “to assist the Borrower in sustaining natural resource management, in particular on crop and range land, and improving productivity attained with greater involvement of resource users in development programs.”2 2 A similar statement of objectives is presented in the project appraisal document (p. 1): “(t)he objectives of the project are sustainable natural resources management, in particular of crop and rangeland in severely degraded zones, and agricultural productivity improvements, attained with greater involvement of resource users in development programs.” The project appraisal document also outlines the following strategies for achieving project objectives: “(a) improve natural resources management, preserving resources and, where feasible, helping to reverse degradation, by developing integrated land management practices, controlling land erosion and introducing new technologies to increase on-farm income; (b) improve rural communities' standard of living and promote better-targeted and more durable investments by implementing Participatory Development Plans (PDP) that strengthen partnership between the administration and local communities and increase 4 RELEVANCE OF OBJECTIVES 2.5 The project’s objectives are relevant to the country‘s development needs as outlined in the government’s five-year Economic and Social Development plans. They are also aligned with the Bank‘s Country Assistance Strategy at project approval (1996–1999), and remain relevant to priorities identified in the current Interim Strategy Note (2013–2014). 2.6 Tunisia’s Economic and Social Development plans have consistently highlighted the need for preserving natural resources and enhancing the local population’s socio-economic development. The 9th Plan for Economic and Social Development (1997–2002) included the goals of involving all stakeholders in development choices, and promoting environment and natural resources preservation. The 10th Economic and Social Development Plan (2002– 2006) placed high priority on the socio-economic development of the rural populations and sustainable natural resources management. The 11th Economic and Social Development Plan (2007–2011) called for the rationalization of natural resource use as a means to transform the agricultural system and diversify the productive base. The 12th National Economic and Social Development Plan (2010–2014) also highlights the need for improving the socio- economic conditions of the rural population, and promoting better protection and management of natural resources. Tunisia is also a party to international conventions on land degradation and desertification (UNCCD), climate change (UNFCCC), wetlands (Ramsar) and biodiversity (CBD), further underscoring the importance of sustainable natural resources to national priorities. 2.7 The World Bank’s Country Assistance Strategy for 1996–1999 promoted the sustainable management of land and water resources, decentralizing decision-making while strengthening the capacity of institutions to monitor and evaluate project impact; and improving the standard of living of the rural population. The goal of social and economic inclusion in the current Interim Strategy Note calls for greater participation of local authorities and local communities in the design of economic policies, in decision-making and in delivery of public services; improved access to basic services for underserved communities; better management of natural resources; and participatory community development in rural communities. 2.8 Relevance of the objectives is rated substantial. DESIGN 2.9 The project was designed as the first phase of a program to integrate the participatory approach to enhancing natural resources management and improving agricultural productivity into the Ministry of Agriculture’s Regional Agriculture Development Centers. The project would pilot the approach within three Regional Agriculture Development Centers, representative of Tunisia’s agro-ecological diversity. A second phase project was envisioned to consolidate the lessons of this project and scale up implementation of the IPA in CRDAs nationwide. ownership and responsibility on the part of beneficiaries; and (c) support Ministry of Agriculture decentralization by introducing the participatory approach at the CRDA level, and provide tools to enhance policy analysis and improve project planning and implementation for sustainable natural resources use.” 5 Components 2.10 The project had two components. 2.11 Component I: Natural Resources Management Operations (Estimated cost US$44.4 million; Actual cost US$32.5 million):  Sub-component 1.1. Participatory Development Plans (Estimated cost US$38.0 million; Actual cost US$29.4 million). This sub-component financed Participatory Development Plans (PDPs) prepared by local communities to address their priorities for key small-scale rural infrastructure investments. It also supported, on a demand basis, investments in soil and water conservation works on private and community lands, agricultural and pastoral development, rehabilitation of small-scale irrigation schemes, rural infrastructure works, and women’s support activities.  Sub-component 1.2. Sub-sectoral Investments (Estimated cost US$6.4 million; Actual cost US$3.1 million). This sub-component supported implementation of the Government's routine soil and water conservation strategy by funding works to prevent soil erosion and recharge groundwater on public lands located upstream of participating communities lands. Three types of works were financed under this component: hill-ponds, water recharge works and spate irrigation works. 2.12 Component II: Institutional Strengthening (Estimated costUS$6.9 million; Actual US$2.9 million):  Sub-component 2.1. Project Management. This sub-component funded the project implementation unit.  Sub-component 2.2. Institutional Development at CRDA Level. This sub-component provided training and technical assistance to strengthen the capacity of the Regional Agriculture Development Centers in participatory management. Geographic Scope of the Operation 2.13 The project was carried out in the governorates of Jendouba, Kasserine and Medenine. The project areas were selected based on a study of natural resources management problems in Tunisia. The criteria for selection were: (a) a high level of resource degradation; (b) a low degree of development; (c) the potential for sustainable agricultural productivity increases and production diversification; and (d) capacity of the Regional Agricultural Development Centers. 2.14 The project zones corresponded to the three major agro-ecological zones in Tunisia: the humid and diversified North, the dry agro-pastoral Center, and the arid pastoral South. Project interventions were concentrated in severely degraded zones within each agro- ecological zone. The project areas are on the lower (poverty) end of the national income spectrum, with relatively high population density, high land fragmentation and high labor emigration. 6 Implementation Arrangements 2.15 The Project was implemented through a multi-tiered arrangement. The implementing agency was the Ministry of Agriculture. Implementation coordination was the responsibility of the Ministry of Agriculture's General Directorate of Finance, Investments and Professional Organizations (DGFIOP). The majority of direct project implementation, however, was undertaken at the regional level in the Regional Agriculture Development Offices and at the local level by local Development Committees with the support of community animators hired by the CRDA. 2.16 At the central ministerial level, a central Project Implementation Unit (PIU) was created in the Ministry of Agriculture's General Directorate of Finance, Investments and Professional Organizations (DGFIOP). The PIU was responsible for overall coordinating responsibilities for project implementation, liaising with the three CRDA's Project Coordinators. The project coordinator was also charged with implementation of a parallel EU-financed Integrated Rural Development and Natural Resources Management Program that co-financed some project activities. 2.17 At the regional level, a Project Coordinator was designated in each of the three CRDAs and was responsible for project implementation. The project coordinator reported directly to the Director General of the Regional Agricultural Development Office. Project implementation was based on annual contracts, signed by the Project Coordinator and Community Representatives, after regional annual programs had been accepted by the CRDA Advisory Committee which included representatives from the Communities. 2.18 At the local level, project management was carried out by local Development Committees and animators. Participation was catalyzed by community workers, or "animators" who coordinated the involvement of technical specialists and led local communities through an interactive process to prepare Participatory Development Plans which prioritized local needs, defined actions, specified investments and clarify the responsibilities of the various stakeholders. Multidisciplinary technical teams, trained in participatory methodology assisted communities to identify technically feasible activities. 2.19 Additional project oversight and policy guidance was provided by a National Steering Committee that comprised technical and financial representatives from the Ministries of Agriculture, Environment, Finance, Interior and Economic Development. The Committee was directed by the Minister of Agriculture and the Secretariat was provided by DGFIOP. Monitoring and Evaluation Design 2.20 The project appraisal document identified 12 key performance indicators sub-divided into three categories that corresponded to three elements of the project objective: (a) sustainable management of natural resources; (b) improved living conditions for rural populations; and (c) institutional support. Seventeen incremental indicators were also identified. Each indicator included incremental targets for 3 years and 5 years of project implementation. Targets were customized by project region. Baselines were to be determined during the preparation of community development plans. However, the key performance indicators and the intermediate indicators were primarily output indicators and did not 7 sufficiency capture the project’s intended results. No indicators were established to measure changes in soil loss, water recharge or siltation or to measure changes in community empowerment or social capital — important aspects of what the participatory approach aims to achieve. 2.21 A pilot M&E system was to be developed under the project’s Institutional Strengthening Component to enhance the monitoring and evaluation capacity of the CRDA. The project appraisal document envisioned that this would be a locally based system capable of assessing the project’s economic and environmental impacts and would enhance the CRDA’s capacity for planning, policy design, and analysis. The system was to comprise a database of natural resources in the project zones and a set of panel interviews to provide statistical information on the population in the three zones. The system would be based on a standardized framework to allow for collection of information at the community level and facilitate its dissemination among the CRDAs and between the CRDAs and the Central implementation level. In addition, the EU was to finance the establishment of a Permanent Information and Evaluation system that would be used by the PIU to monitor the sustainable management of natural resources. The system would include a geographic information dataset and a permanent household sample available for periodic surveys. Development of the system was to be financed through a separate EU-financed integrated rural development project implemented in parallel but managed by the same project implementation unit. RELEVANCE OF DESIGN 2.22 The project included a clear statement of objectives and component activities that were relevant to achieving the project objectives, but there were some gaps. Project activities in support of agriculture focused on addressing the limiting factors of production such as water availability and land degradation, but were not matched by support for addressing market access constraints. Input supply and access to credit were also insufficiently addressed. Livestock development activities were excluded from the project, even though small-scale traditional livestock is particularly important to livelihoods in Southern Tunisia. The scope of the objectives was overambitious relative to its budget and time frame, based on IEG interviews. 2.23 The relevance of design is rated Modest. Implementation 2.24 The project was approved on May 13, 1997, and became effective on October 15, 1997. The project’s closing data was extended by 12 months, from June 30, 2003, to June 30, 2004, to compensate for a lag in implementation of the EU-financed component for staff training. The Project’s objectives and components were not changed during the course of implementation. PLANNED VS. ACTUAL EXPENDITURE BY COMPONENT 2.25 The total project cost estimated at appraisal was US$51.3 million equivalent. Actual project costs were US$35.4 million equivalent, almost one-third less than expected, primarily 8 because the Tunisian Dinar depreciated by 27 percent relative to the U.S dollar during implementation.3 2.26 Project costs were to be financed by an IBRD credit of US$26.5 million (52 percent of total project costs), a government contribution of US$16.6 million (32 percent of total project costs), EU co-financing of US$ 1.7 million4 (3 percent of total project costs), and beneficiary contribution of US$6.5 million (13 percent of total project costs). At project closure US$20.9 million equivalent of the IBRD loan had been disbursed and US$0.95 million equivalent was cancelled. The Government counterpart contribution was US$9.32 million equivalent (26 percent of total project costs compared to 32 percent forecast at appraisal).5 Beneficiaries contributed US$4.98 million (14 percent of total project cost compared to the 13 percent forecast at appraisal). The EU provided US$0.2 million equivalent (1 percent of total project cost compared to 3 percent forecast at appraisal). The EU’s contribution did not occur as envisaged due to delays in the development of the training program that the contribution was to finance. The beneficiaries contributed more than planned, due a change in government policy during the project implementation period, which raised the minimum beneficiary contribution to all development projects. 2.27 Table 1 summarizes the total project costs as planned, versus actual disbursements against each of the components. IMPLEMENTATION EXPERIENCE 2.28 Project implementation was challenged by several factors. 2.29 The EU financed component to train CRDA staff in the participatory approach was delayed which contributed to implementation delays and affected the quality of the initial participatory development plans. During negotiations, it had been decided to allocate the equivalent of US$1.7 million from an EU-financed rural development project which was about to begin for a training program to enhance the capacity of staff of both projects. Development of the training program, however, was delayed by two years. Consequently the CRDA's multi-disciplinary teams that facilitated development of the first round of participatory development plans were not adequately prepared. Supervision documents indicate that the Participatory Development Plans and associated Annual Contracts prepared at this time were too ambitious in relation to the budget available for their implementation. The initial analysis and diagnosis feeding into these plans was inadequate and most plans did not have sufficient performance indicators. Annual Contracts often took more than a year to prepare and there were cases of overlap (sometimes by more than 1 year) within the same local community, which risked depleting the Project’s budget before completion. 3 The loan was made in Euros. Euro: € 22.9 million equivalent to US$26.5 million at appraisal. Total Loan disbursements, net of commitments before Project closing, amounted to € 21.9 million or 96% of the Loan. 4 During negotiations, it had been decided to allocate the equivalent of US$1.7 million to the NRMP -- from the EU-financed Rural Development and Natural Resource Management Program to cover a training component for staff of both projects. 5 IEG was not able to find the reason why the percent of project costs covered by the IBRD loan were slightly higher than anticipated at appraisal and the government contribution was less. 9 Table 1. Project Costs by Component (in US$ million equivalent) Appraisal Percentage Actual Amount Components Estimate of (US$ millions) (US$ millions) Appraisal A. Natural Resource Management Operations 1. Participatory Development Plans 11.70 8.80 75.2 - SWC Works - Agricultural and Pastoral Development 10.30 2.90 28.2 - Rehabilitation of Small-Scale Irrigation 3.80 6.10 160.5 Schemes - Rural Infrastructure 11.50 7.80 67.8 - Support to Women's Activities 0.70 3.80 542.9 2. Water Recharge and Protection Works 4.20 1.60 38.1 - Hill Ponds - Water Recharge Works 1.30 1.30 100.0 - Spate Irrigation Works 0.90 0.20 22.2 B. Institutional Strengthening 1. Project Management (PIU) 2.00 0.30 15.0 2. CRDA Institutional Support 4.90 2.60 53.1 Total Project Costs 51.30 35.40 Source: World Bank 2004 2.30 The project faced multiple staffing challenges. Project files indicate that there were delays in establishing the Project Implementation Unit and Regional Coordination Units and staffing them and some staff planned at appraisal were never recruited. The national level project coordinator only held a part-time position with the project and did not have sufficient leverage over project staff in the CRDA to play an effective coordination role, according to IEG interviews. Two of the three regional coordinators did not have sufficient seniority to play an effective role within the CRDA or in collaborating with the CRDA commissioner. The project completion report notes that there were delays in assigning technical staff in the CRDA to specific project activities. No recruitment was carried out and the existing technical staff within the CRDA who were eventually assigned to the project were responsible for carrying out their regular duties in addition to project responsibilities. Some staff left mid- project and were not replaced. The training and programming specialist in the project implementation unit retired in 2002 and the regional staff in charge of participatory programming in Jendouba and Kasserine left their positions in 2001 and 2003 to find more permanent employment. Due to a nationwide hiring freeze for non-technical government staff, most of the “animators” hired to mobilize communities were recruited on a short-term contractual basis. Many left the project for more secure employment. Supervision reports also show that there were several changes in high-level officials such as provincial governors and CRDA commissioners during project implementation that caused fluctuations in the official support structure. 10 2.31 Drought. Project areas were affected by four consecutive years of drought (1998– 2002) which reduced agricultural and pastoral productivity gains and limited soil conservation benefits. 2.32 Some activities were constrained by a lack of access to credit and changes in the minimum community contribution rate. At appraisal it was assumed that smallholder access to credit would be guaranteed on a declining basis in accordance with the Government’s budget management program. In practice small farmers and women in the project areas were not able to access credit because they were not able to provide a certificate of ownership or other bank guarantees. The projects efforts to support the promotion of income-generating activities were constrained by the absence of micro-credit organizations in some regions.6 In addition, during implementation the beneficiaries’ contribution rates were revised upwards due to a change in government policy that raised the minimum beneficiary contribution to all development projects to bring them in line with the more stringent requirements of the National Agricultural Bank’s Special Fund for Agriculture and Fisheries Development. Rates for the rehabilitation of small-scale irrigation increased from 40 to 70 percent. Supervision reports indicate that the higher rates were a particular constraint to the construction of irrigation basins and water reservoirs in the region of Medenine. IMPLEMENTATION OF MONITORING AND EVALUATION 2.33 The project’s completion report indicates that project outputs and impacts were not adequately monitored and progress reports did not cover these issues. It also indicates that the 12 key performance indicators could not be tracked. The monitoring system was designed by a consulting firm in 1999 but it was not used as the project implementation unit found it too cumbersome to operate. A second monitoring system was established with a simpler design, but it was implemented too late in the implementation period (2003) to be useful. In the meantime the Regional Coordination units monitored the activities implemented in their jurisdiction with excel spreadsheets that were formatted differently, which created data migration problems that could not be corrected. A consolidated data system was introduced across all regions in 2002. 2.34 The pilot M&E system for the CRDA was not developed. The Permanent Information and Evaluation system was not established by the EU. The EU decided to delay preparation until it completed a study it was conducting on natural resources that was expected to generate satellite images and other information relevant for the system. 2.35 The project completion report notes that the lack of a sound M&E system hindered the PIU’s coordination efforts and the documentation of project impacts. There is little evidence that the monitoring activities that were carried out contributed to strengthening institutional capacity. Preparation documents for a second phase of the project reveal that the monitoring system that was established at the regional level did not fit the reporting requirements of the CRDA and were not maintained. 6 The Tunisian National Bank only provides loans in excess of 2000 Tunisian Dinars. Loans below this threshold are met by local development associations (NGOs authorized to grant micro-credits) and these associations were not active in all regions. 11 SAFEGUARDS COMPLIANCE 2.36 The project was classified as category B under the Bank‘s environmental and social safeguards framework and also triggered the policy on Forests (OP 4.36). A field-based environmental review was carried out during preparation that anticipated that the environmental impact would be positive. An environmental mitigation plan was also developed during project preparation, and specific criteria and methodology were established to select the works which would be constructed to prevent soils erosion and recharge groundwater. Supervision reports indicate that compliance with both policies was satisfactory throughout project implementation. Achievement of the Objectives OBJECTIVE 1: SUSTAINABLE NATURAL RESOURCES MANAGEMENT, IN PARTICULAR OF CROP AND RANGE LAND IN SEVERELY DEGRADED ZONES 2.37 This objective was pursued through investments in soil and water conservation works and the establishment of pastoral plantations carried out with the active participation of the populations concerned on private agricultural land and community lands. Erosion control and groundwater recharge investments were also supported on state land adjacent to participating communities’ territories. Works on state land were included because they supported the Government's national soil and water conservation strategy and were expected to have positive spillover effects on the communities downstream. The aim of the soil and water conservation investments was to arrest and — where feasible — reduce or reverse degradation, by controlling land erosion and increasing water infiltration. The establishment of pastoral plantations was expected to reduce pressure on rangelands by planting of fodder trees (acacias, cactus). OUTPUTS 2.38 Soil and water conservation works. As indicated in Table 2 most of the appraisal targets for soil and water conservation works on both public and beneficiary lands were met or exceeded. The number of irrigation and water recharge works established on public lands in Jendouba and Kasserine exceeded 200 (versus the respective appraisal targets of 47 and 118). There was a shortfall in the works established in the governorate of Medenine (only 15 were established compared to the target of 95) due to drought. 2.39 Pastoral plantations. The establishment of pastoral plantations fell short of appraisal estimates in all three regions due to an extended period of drought. At appraisal it was anticipated that the area covered by pastoral plantations would increase by 5 percent in Jendouba, by 4 percent in Kasserine and by 3 percent in Medenine. The actual increase in plantation areas were .06, .01 and .005 respectively. OUTCOMES 2.40 There is no outcome level data to demonstrate the impact of the project investments on the natural resource base. Investments in soil and water conservation works aimed to reduce runoff and sedimentation of dams, stabilize and reverse erosion, restore soil fertility and enhance groundwater recharge. None of these changes were measured. 2.41 Achievement of this objective is rated modest. 12 Table 2. Sustainable Natural Resources Management Achievements Appraisal Estimates (%) Completion Rates (%) JENDOUBA KASSERINE MEDENINE JENDOUBA KASSERINE MEDENINE Number of spate irrigation and water +47 +118 +95 >200 >200 +15 recharge works completed (public land) Soil & Water Conservation areas +13 +24 +16 +46 +22 +52 (private and community land) Area of soil and water conservation treated lands as a percentage of - - - 83 29 80 the total area of eroded land Pastoral plantations +5 +4 +3 +0.6 +0.1 +0.05 Source: Project Completion Report Natural Resources Management Project OBJECTIVE 2: IMPROVING AGRICULTURAL PRODUCTIVITY 2.42 According to the project appraisal document this objective was pursued through the rehabilitation of small-scale irrigation systems and the provision of agriculture extension efforts including establishment of tree crops. The project also supported the construction of water systems, new roads, the rehabilitation of existing rural feeder roads, and road maintenance. The project documents note that improving rural feeder roads was expected to facilitate access to inputs, provisions, and markets for agricultural products and contribute to agriculture development for rural communities. It was also expected to facilitate access to services such as health, education, administrative, and agricultural services. Infrastructure investments were also expected to act as an incentive for the communities to participate in more efficient natural resource management and anti-erosion measures to which they might not otherwise have been sensitive (e.g. closing of rangelands during deferred grazing periods). Income-generating activities for women were also supported (rabbit and poultry breeding, beekeeping, handicrafts) which were intended to contribute to the family livelihood by generating income or contribute to household consumption. OUTPUTS 2.43 Agriculture production investments. IEG was unable to obtain systematic documentation of the agriculture production activities that were financed and their resulting outputs. Neither project supervision documents nor the project completion report indicate the number of small-scale irrigation systems that were constructed, what kind of services were provided to farmers through extension efforts or how many farmers benefitted from these activities. 2.44 Rural infrastructure. Data reported in the project completion report for rural infrastructure improvements show that increases in communities’ access to roads and potable water were below expectations. The project appraisal document anticipated that by project closure over 90 percent of the households in project areas would have access to rural roads 13 and 100 percent of the households in project areas would have access to potable water. The project completion report indicates that only 25 percent of households in Jendouba, 75 percent of households in Kasserine and 100 percent of households in Medenine had access to rural roads. Only 22 percent of households in Jendouba, 75 percent of households in Kasserine and 95 percent of households in Medenine had access to potable water by project closure. However, the project did not report on the number of water sources or kilometers of rural roads constructed by the project and no control groups were established to determine the extent to which the increase in communities’ access to rural infrastructure was due to project activities. The project completion report concludes that appraisal estimates of access to rural infrastructure were over-optimistic relative to the project’s budget. Table 3. Rural Infrastructure Achievements Appraisal Estimates (%) Completion Rates (%) JENDOUBA KASSERINE MEDENINE JENDOUBA KASSERINE MEDENINE % of households with access to rural >90 >90 >90 25 75 100 roads % of households with access to 100 100 100 22 75 95 potable water Source: Project Completion Report Natural Resources Management Project 2.45 Women’s Income-Generating Activities. Training and basic materials were provided to develop micro-projects in beekeeping, rabbit breeding, poultry raising, cottage industry weaving, vegetable gardening. IEG was unable to obtain systematic documentation of the number of income-generating activities financed by the project. The project completion report indicates that demand for these projects exceeded forecasts, but successfully developing these projects was limited by a lack of sufficient micro-credit. It was also reported that the limited scale of these activities and sales of these products reduced their benefits. Reports from the beneficiary workshop held at the end of the project and IEG interviews indicate that rabbit breeding was unsuccessful particularly in the southern most governorate, which beneficiaries attributed to the hot climate and high cost of inputs. OUTCOMES 2.46 Agricultural production investments. The project reported on changes in wheat and olive yields but the data were based on national statistics for the entire governorate and not specific to project areas. According to the data presented, wheat yields met or exceeded the appraisal estimates in two governorates, but were only half of expected in the third. Olive yields were below expectation in all three governorates. The lag in productivity gains was attributed to the four years of drought that occurred during project implementation. In the absence of control groups, it is not possible to determine the extent to which project activities contributed to these results. Table 4. Agriculture Productivity Achievements Appraisal Estimates (%) Completion Rates (%) 14 JENDOUBA KASSERINE MEDENINE JENDOUBA KASSERINE MEDENINE Wheat Yields (ton +0.8 +0.5 +0.2 +0.4 +0.5 +0.05 per hectare) Olive Yields (ton +0.8 +0.5 +0.2 +0.6 +0.3 -0.2 per hectare) Source: Project Completion Report Natural Resources Management Project 2.47 Participants of a stakeholder workshop at project closure reported that the rehabilitation of small-scale irrigation schemes and the water-saving measures introduced by the project contributed to an increase in the development of agricultural land, intensification of agricultural production systems, and a reduction in farmers' workloads. No quantified evidence has been documented to substantiate these claims. 2.48 Rural infrastructure. Anecdotal evidence from stakeholder workshop and IEG interviews report that the construction of rural roads increased access to health facilities and markets, and helped create new jobs by facilitating beneficiaries’ access to neighboring regions where day-labor opportunities existed, but there is no quantified evidence to substantiate these claims. 2.49 Women’s income-generating activities. The project completion report indicates that income-generating activities were put in place too late in the project cycle to see an impact by project closure and there has been no follow-up monitoring since the project’s closure. IEG was unable to obtain documentation on the number of income-generating activities that continued to operate or the amount of income they generated at the time of the assessment mission. 2.50 Achievement of this objective is rated modest. OBJECTIVE 3: GREATER INVOLVEMENT OF RESOURCE USERS IN DEVELOPMENT PROGRAMS 2.51 This objective was pursued by implementing the participatory approach through the Regional Agriculture Development Offices. This was expected to strengthen partnerships between the local administration and local communities, and increase ownership and responsibility on the part of beneficiaries. OUTPUTS 2.52 The project implemented the participatory integrated development approach in three regional agricultural centers as planned. Most of the output targets related to implementing the approach were met or exceeded: 101 Participatory Development Plans were prepared under the project, just shy of the appraisal estimate of 114. However, the number of households covered by these plans (12,800 households) far exceeded appraisal estimates (7,000 households) because the participating communities had higher population densities than anticipated. The number of annual contracts financed to fund the activities prioritized in the participatory development plans exceeded appraisal estimates — 352 as opposed to 234 projected at appraisal. The project’s efforts to develop partnerships leveraged an additional DT 16 million from national financing schemes that had not been envisaged at appraisal. An 15 additional indication of community engagement in development process is that the beneficiary share of the cost of soil and water conservation was in line with what had been forecast in almost all the regions. Table 5. Sustainable Natural Resources Management Achievements Appraisal Estimates (%) Completion Rates (%) JENDOUBA KASSERINE MEDENINE TOTAL JENDOUBA KASSERINE MEDENINE TOTAL Number of PDPs 44 52 18 114 36 47 18 101 completed Number of Annual 86 102 46 234 138 145 69 352 Contracts Completed % of Beneficiary contribution to the cost of soil +20 +10 +10 10 10 10 and water conservation works Source: Project Completion Report Natural Resources Management Project OUTCOMES 2.53 The outcome level achievement of this objective is assessed by the extent to which communities have been empowered and continue to engage in the local development process. The project did not assess changes in community empowerment or enhanced community capacity. Although the project successfully engaged communities in the local development planning process while the project was active, the evidence on the sustainability of the integrated participatory approach among these communities is meager. The community-level development committees that served as the interface between the communities and the local development process were project-driven entities. Most did not have formal legal status which meant that they did not have the legitimacy to engage on their own with government agencies involved in the local development process. Consequently most ceased to operate following project closure. Evidence that the project enhanced the capacity of the Regional Agricultural Development Offices (CRDA) to continue implementation of the integrated participatory approach is also weak. The human resource capacity of the CRDA to implement the IPA was strengthened through staff participation in the implementation of the project. However, as noted in paragraph 2.30, the project relied heavily on consultants hired on a part time basis, many left their post due to job insecurity and permanent staff who retired were not replaced. It was also anticipated that by the end of the Northwest Mountains and Forest Areas project, the CRDA’s would ODESYPANO’s role in promoting the IPA in the Northwest part of the country. An assessment conducted at the end of that project, however, found that the CRDA lacked the budget and human resource capacity to play this role. A follow-on project is ongoing aimed at creating more permanence of the IPA within the CRDA, but there was a lengthy delay in the project’s approval and it has faced a number 16 of implementation setbacks. Implementation of the follow-on project was at an incipient stage at the time of the IEG assessment mission. 2.54 Achievement of this objective is rated modest. Efficiency 2.55 At appraisal an economic rate of return for the project as a whole was estimated to be 13.7 percent over 30 years. This calculation took into account all project costs and operation and maintenance costs beyond the end of the project. In terms of benefits, the calculation only included benefits expected to accrue through increased agriculture and fodder production, soil conservation on private and community lands and increased availability of water for both consumption and irrigation. The calculation did not include an estimation of the benefits from soil conservation and water recharge works carried out on state land outside of the communities, rural roads and women’s support activities. These were considered as indirect benefits that were difficult to quantify. 2.56 The economic rate of return was recalculated at project completion at 18.5 percent, using the same methodology employed at appraisal. The project’s completion report attributes the higher ERR at project closure to participation of a larger number of farmers than originally anticipated and to cost savings that resulted from the use of small independent contractors as opposed to large consulting firms. 2.57 It is likely that the ERR calculation at closure overestimates the project’s return. The project completion report notes that benefits included in the ERR calculation were mainly derived from increased crop and fodder production and these were the activities most constrained by the drought. Moreover, no control groups were established to allow for an accurate estimation of incremental net economic benefits. 2.58 In terms of the efficient use of resources in project implementation, actual project costs in US dollar terms were almost one-third less than expected but that was due to the 27 percent deprecation of the Tunisian Dinar during implementation. The loan was almost fully disbursed yet the number of households served by the project (12,800) greatly exceeded appraisal estimates (7,000). There were implementation delays in the initial years of implementation due to the lag in implementation of EU-financed training component and the slow establishment and staffing of the central and regional project coordination units. The project closed a year later than initially planned. 2.59 Efficiency is rated modest. Ratings OUTCOME 2.60 The overall outcome of the project is rated moderately unsatisfactory. The project development objective was appropriate to the needs of Tunisia’s rural sector but there were gaps that weakened the relevance of design to achieving the project’s objectives. Achievement of the project’s objectives was modest due to limited evidence of outcome level achievements. Efficiency was also modest. 17 RISK TO DEVELOPMENT OUTCOME 2.61 The development gains made by project closure were at an incipient stage and in need of consolidation. Although a follow-on project was developed, there was a 6 year delay between closure of the project and initiation of the follow-on phase.7 In the interim many of the capacity building gains in the local community and the Regional Agricultural Centers have eroded. Interviews indicate that although the government devoted a small amount of bridge financing to the CRDA of approximately 2 million Tunisian dinars, this was not sufficient to maintain the structure in the interim. The project’s institutional memory has been lost because very few staff remains from the first phase of the project. The community development committees lacked formal legal status and most ceased to operate following the project. Many of the local contractors hired to carry out work prioritized through annual contracts were unable to find employment following the project. Following the revolution many of the formally recognized producer organizations have broken down as communities viewed them as arms of the administration. Tensions have also risen between communities and the Regional Agriculture Development Centers and other government entities. CRDA staff also reported to IEG that they did not have the HR capacity to engage as intensively with communities as required. 2.62 The sustainability of the investment infrastructure built by the project depends on it is being maintained. Government agencies are responsible for maintenance of the soil and water conservation works completed on government land. Those investments visited by the IEG mission were in good working order. Investments on private and communal land are the responsibility of beneficiaries. The IEG mission visited a small non randomly selected sample of farms where water harvesting and irrigation systems were constructed that were in working order and farmers indicated that the costs of maintaining these investments were minimal and within reach of their income. However, it was not clear if all beneficiary farmers were equally capable of covering the expense of maintaining their investments. In addition, in the case of some infrastructure investments such as roads, there was a lack of clarity as to who would be responsible for maintenance. In one community visited by the IEG mission, the drainage channels on the side of the road were full of silt and garbage. Drainage on the sides of roads should be kept clear so they are not prone to water damage. Community members and government officials disagreed as to who was responsible for maintenance. The 7 The first phase of the project closed in June 2004. The follow-on project was approved in December 2010. It became effective in February 2011. During the IEG mission in October 2012, implementation was at an incipient stage. The project was restructured in 2013. IEG was unable to find the reasons for the long delay between the close of the first operation and approval of the second. The following factors contributed to the delays in start up and to poor implementation progress following loan approval: (i) the disturbances and, at times, security concerns associated with the ongoing socio-political transition following the early-2011 Tunisian revolution; (ii) the weakening or dissolution of several local and regional organizations (ADGs, RDCs, and LDCs) representing key institutional actors or stakeholders involved in the project; (iii) the new challenges of involving the local population in the participatory process, stemming from security concerns in the field for the local staff of the Ministry of Agriculture combined with their lack of experience in dealing with social tensions and conflicts; (iv) the frequent turnover of staff of the RCADs and other partner institutions at the local or regional levels; (v) the inadequate staffing of the project units and excessive delays in the appointment or recruitment of personnel dedicated to project implementation at the regional level; (vi) the relative complexity of the project design and the need for adjustments to the post-revolution environment; and (vii) the weaknesses in the overall administration of the project and insufficient capacities and mastering of project operational procedures, notably in procurement. 18 community members claimed that they did not have access to the equipment required to clear sedimentation. 2.63 The risk to development outcome is rated Significant. BANK PERFORMANCE Quality at Entry 2.64 Project preparation included a study of natural resources management problems in Tunisia, consultations with targeted communities and took into account lessons learned in earlier Bank-financed integrated rural development project carried out in country. The main risks were identified at appraisal and adequate mitigation measures were set forth, with two exceptions. The risk that co-financing would not materialize on time was not identified and the measures to mitigate the risks from potential drought proved to be insufficient. A formal framework to guide the engagement with co-financing and implementation partners was not put in place. As a result interactions between these partners was ad-hoc. On balance, the Bank’s performance in ensuring quality at entry is rated moderately satisfactory. Quality of Supervision 2.65 Supervision reports indicate that the Bank carried out supervision missions at regular and frequent intervals and fielded teams with a good skills mix. Throughout implementation supervision reports flagged issues of concern and the Bank team provided guidance to improve implementation. The Bank’s supervision team facilitated implementation by limiting the number of Participatory Development Plans prepared in each year to improve the quality of the plans prepared and mitigate the threat of depleting the project’s budget before project completion. The Bank also demonstrated flexibility in extending the project closure date to make up for the late start of co-financed activities. However, IEG interviews indicate that changes in task managers during implementation adversely impacted the quality of some supervision missions and procurement support, as each new task manager needed time to come up to speed on the projects, which delayed approval in some cases. In addition the Bank team should have played a more proactive role in ensuring implementation of a functional M&E system. Accordingly, supervision is rated moderately satisfactory. 2.66 Taking into account the ratings for quality at entry and supervision, the overall Bank performance is rated moderately satisfactory. BORROWER PERFORMANCE Government Performance 2.67 The government showed a high degree of ownership and commitment to the project through its active engagement in project preparation and financing commitments. However, the government could have played a more active role in trying to resolve the delay in availability of the EU co-financing and lack of solid M&E system. Overall government performance is rated moderately satisfactory. 19 Implementing Agency Performance 2.68 The project implementation unit was proactive in ensuring oversight and took the initiative in addressing many implementation challenges. Supervision reports show that the central project implementation unit conducted over 30 support missions to project sites and organized regular meetings with the three regional coordinators, which facilitated harmonization of project activities across the three regions and served as an institutional mechanism for sharing information and ideas across regions. The project implementation unit was proactive in implementing a temporary training program to support the CRDAs at the beginning of the project to compensate for delays in the EU-financed training component. Project implementation was also aided by the PIU’s preparation of thematic reports (such as the evaluation of small contractors, annual contracts, and development committees), methodological tools (guides, manuals, etc.) and other supporting documentation. The implementing agency’s initiative in facilitating implementation was further demonstrated by: (i) its approval of the subdivision of contracts to allow CRDAs to utilize a larger number of small local contractors rather than large firms (ii) the authorization of a budget advance procedure to facilitate implementation of the integrated participatory approach; and (iii) its mobilization of parallel financing. 2.69 Shortcomings in implementing agency performance include delays in establishing and staffing the PIU and Regional Coordination units, failure to replace project staff who left their positions during implementation, and the lack of a sound M&E system which hindered the project implementation unit’s coordination efforts and the documentation of project impacts. On balance implementing agency is rated moderately satisfactory. 2.70 Taking into account the ratings for government and implementing agency performance, the overall Borrower Performance is rated moderately satisfactory. MONITORING AND EVALUATION 2.71 Design. Performance indicators were identified at appraisal along with targets but there were gaps in key output indicators. 2.72 Implementation. Project outputs and impacts were not adequately monitored and progress reports did not sufficiently cover these issues. The permanent evaluation system expected to be financed by a parallel project did not materialize. Several stakeholder workshops were conducted at project completion that provided anecdotal information on project’s progress as well as its implementation challenges. 2.73 Utilization. The lack of a sound M&E system hindered the PIU’s coordination efforts and the documentation of project impacts. 2.74 Overall, the quality of M&E is rated negligible. 20 3. Northwest Mountainous and Forestry Areas Development Project (2002–2009) Background and Project Context 3.1 The Northwest region of Tunisia is of significant national importance and faces distinct development challenges. It is considered to be the “water reservoir” of the country because approximately 75 percent of the national water supply originates from its watersheds. The region is highly vulnerable to soil erosion and degradation due to its steep slopes and periods of drought followed by intense seasonal rainfall. It is estimated that 60 percent of land in the region is eroded, contributing to the loss of 13,000–23,000 ha of topsoil each year and resulting in siltation of water reservoirs and a decrease in dam storage capacity. The Northwest is also one of the most isolated and disadvantaged regions in the country. Mountains and forest areas cover about 60 percent of the region and the socioeconomic indicators for its population are among the lowest in the country. Infrastructure and public services in the mountain areas of the region are insufficient (agricultural extension and research, road construction/maintenance, health and education) as they are costly to implement and difficult to maintain (World Bank 2010b). 3.2 The Northwest Mountains and Forest areas project was the third in a series of World Bank funded projects aimed at addressing the natural resource degradation and rural development in mountain areas of the northwest region using the Northwest Forestry and Pastoral Development Agency (ODESYPANO) as the implementing agency. ODESYPANO was created in 1981 to protect natural resources in two major watersheds in the Northwest region, through the construction of rural infrastructure and implementation of anti-erosion measures. 3.3 The first project, the North West Rural Development Project (implemented from 1982–1989), employed a top-down approach with technical criteria for the management of watersheds. The project successfully provided communities with basic infrastructure, including water supplies and schools, and raised consumption levels. But the project’s efforts to slow down resource degradation had limited results. Almost all measures on private land were resisted. Soil conservation measures were only accepted on fallow land where farming could not be carried out. The project did not persuade farmers to change their cropping and livestock practices. The second project, the Northwest Mountainous Areas Development Project (1994–2001) had a poverty alleviation and natural resource management focus and used a participatory approach that emphasized community organization and training to increase the capacity of local communities to participate. 3.4 The third operation (under review) was approved to improve and expand the use of the integrated participatory approach and further strengthen the communities’ capacity to become effective planners of their development in the medium to long term. At the end of the project, responsibility for organizing communities and developing their capacity to participate in local development via the integrated participatory approach was to be transferred to the Ministry of Agriculture’s Regional Agriculture Development Offices (CRDA). It was believed that the CRDA would have a comparative advantage over ODESYPANO because they are permanent structures and are more integrated with the 21 various regional work programs relevant to addressing the activities identified in participatory development plans. Objectives, Design, and their Relevance OBJECTIVES 3.5 The objectives of the project, as set out in the Lending Agreement (p. 13) were “to increase household incomes and enhance the socio-economic conditions of the population in the project area, while ensuring sustainable management of the natural resources, through the improvement and diversification of the agricultural and pastoral production systems and the promotion of off-farm income-generating activities.”8 RELEVANCE OF OBJECTIVES 3.6 The project’s objectives were relevant to priorities outlined in national strategies and in the Bank’s country assistance strategy document in effect at project appraisal and remain relevant to current strategies. 3.7 Tunisia’s 10th Economic and Social Development Plan (2002–2006), in place at the time of project was prepared, aimed to improve environmental management, including the protection of scarce natural resources. It explicitly called for the promotion of rural development as an instrument for reducing poverty and improving the living standards of rural populations. The 11th Economic and Social Development Plan (2007–2011) included a focus on the promotion of a comprehensive development approach to guarantee sustainable growth and a harmonious balance among economic, social and environmental priorities, and greater diversification of the productive base. The current Government directives outlined in 12th Economic and Social Development Plan (2010–2014) include improving the socio- economic conditions of the rural population, and promoting better protection and management of natural resources in the project area using an integrated participatory approach to community-based development. 3.8 The World Bank’s Country Assistance Strategy for Tunisia (2000–2002) when the project was prepared aimed to strengthen the Government's social agenda through the implementation of environmentally and socially sustainable development, including the protection of scarce natural resources and the continuation of poverty reduction measures. The 2010–2013 Tunisia Country Partnership Strategy, in place at project closure, included a sustainable development pillar that aimed to improve the economic conditions of Tunisia's poorest people in the rural areas of the North West. The project’s objectives remain relevant to the goal of promoting social and economic inclusion in the current Interim Strategy Note. The strategy highlights the need for greater participation of local authorities and local communities in the design of economic policies, in decision-making, and in delivery of 8 A similar statement of objectives is presented in the project appraisal document (p. 3) “(t)he project development objective is to improve the socio-economic conditions of the populations in five governorates covering the mountainous and forested areas of the northwest region, Beja, Bizerte, Kef, Jendouba and Siliana, while ensuring sustainable management of the natural resources. More specifically, the project will aim to increase household incomes through the improvement and diversification of the agricultural/pastoral production systems as well as the promotion of off-farm income-generating activities.” 22 public services including at the rural community level. It also calls for improving access to basic services for underserved communities, including better management of natural resources and participatory community development in rural communities. 3.9 Relevance of the objectives is rated substantial. DESIGN Components 3.10 The project comprised the following components. 3.11 Component 1. Institutional Strengthening for ODESYPANO and Partners (appraisal US$3.01 million; actual US$3.52 million). This component supported capacity building investments aimed at enhancing efficient and cost-effective planning, implementation, and monitoring and evaluation. 3.12 Component 2. Implementation of Pilot Operations (appraisal US$3.25 million; actual US$2.44 million). The component supported three types of pilot operations: (i) formulation and implementation of one Community Development Plan centered on forest dwelling communities in each of the five governorates; (ii) six land consolidation operations over 5,400 hectares; (iii) promotion of micro-projects and micro-enterprises including analytical studies, training, assistance in preparing requests for credit institutions, and support for marketing. 3.13 Component 3. Agricultural and Livestock Development (appraisal US$3.14 million; actual US$2.39 million). The component supported: (i) provision of agricultural extension services; (ii) livestock development activities aimed at improving genetic stock, animal health, and feed quality; and (iii) rehabilitation of small-scale irrigation with water from community wells or existing springs. 3.14 Component 4. Sustainable Natural Resource Management (appraisal US$14.85 million; actual US$16.41 million). The component supported: (i) soil and water conservation works, including stonewalls, anti-erosion plantations, small dikes, and grass strips; (ii) improvement of pasture in range lands and degraded areas; (iii) agroforestry development, through the establishment of plantations such as olive and fruit trees as a complementary measure for mechanical soil and water conservation works. 3.15 Component 5. Improvement of Basic Rural Infrastructure (appraisal US$13.79 million; actual US$19.99 million). The component supported: (i) the rehabilitation and construction of rural roads; and (ii) improved access to potable water through the construction of individual tanks and connections. Geographic Scope of the Operation 3.16 The project was implemented in the five governorates that make up the northwest region of the country: Beja, Bierte, Kef, Jendouba and Siliana. The project areas corresponded only to the mountainous and forest areas in each governorate in accordance with ODESYPANOs mandate. 23 Implementation Arrangements 3.17 The following entities were involved in the project’s implementation. 3.18 The implementing agency for the project was the Northwest Forestry and Pastoral Development Agency (ODESYPANO). It was responsible for project management and coordination, technical implementation, overall fiduciary and safeguard management, and project monitoring and evaluation. ODESYPANO’s central and regional offices supported the communities with preparation, execution and monitoring of community development plans and annual contracts, and promoted partnerships at the regional and local levels to facilitate the integration of Community Development Plans in other partners’ development programs and investment plans. A project coordinator was appointed to prepare periodic work programs and progress reports. The project coordinator was also responsible for coordination with ODESYPANO’s partners and ensuring exchange of information and coordination of common activities. 3.19 National Coordination Committee. An inter-ministerial Committee chaired by the Ministry of Agriculture served as the overall steering committee for the project. Other members of the National Coordination Committee included the Ministry of Development and International Cooperation, Ministry of Finance, relevant General Directors within the Ministry of Agriculture (direction of planning and conservation of agricultural lands). 3.20 Local based (community) organizations included informal Development Committees organized under the project and formally recognized Agriculture Development Groups. These organizations were the representatives of their population and were the institutional interface with all public authorities in terms of planning and programming interventions in the sectors involved. They participated in the preparation of the Participatory Development Plans and implantation of activities through Annual Contracts that were negotiated with the Local Development Council. 3.21 Regional and Local Councils of regional and sub-regional administrations constituted the consultative channels and institutional platforms for the participatory approach at the local level. Regional Councils were chaired by the Governor, the highest regional authority of the governorate. They provided official validation of community development plans and annual contracts. Local councils corresponded to the sub-regional level and were responsible for local development programs and projects in their delegation. Local councils facilitated the mobilization of partners and complementary financing at the local level with a view to ensuring the coherence and coordination of project activities with other local programs and projects. 3.22 Regional Directorates of technical ministries. Regional Directorates participated in preparation financing and execution of community development plans and annual contracts within the limits of their agency mandates. The regional technical agencies involved with the project included the Regional Commissariat for Agricultural Development, Ministry of Equipment, Agricultural Lands Agency, Office of Livestock and Pasture, Directorate for planning and conservation of agricultural lands, General Director of Forests, and the Institute for Agricultural Research and Higher Education. 24 3.23 Other non-government partners. NGOs, Local Development Associations and the Tunisian Bank for Solidarity were involved with development of areas covered by the project. Monitoring and Evaluation Design 3.24 Key performance indicators were identified at appraisal but no targets or baseline values were established. The project’s MIS built on the system that ODESYPANO had utilized in the implementation of prior projects. Provisions were made under the institutional strengthening component for technical assistance to design and implement a management monitoring system. The institutional strengthening component of the project planned to provide technical support to improve upon this system and methodologies to better measure project results. 3.25 Beneficiary communities would monitor project activities in collaboration with animators and subject matter specialists. Beneficiary feedback was to be obtained through surveys, focus group discussions, and field visits with representative population groups. RELEVANCE OF DESIGN 3.26 The project included a clear statement of objectives and the project’s activities were relevant to the achievement of these objectives but there were some gaps related to the income-enhancing objective. The project’s activities in support of agriculture focused on production issues, which were not matched by support to address market access constraints. A positive feature of design was the inclusion of a pilot to address the fragmentation of farms — a significant constraint to agricultural development in the region. A shortcoming of design was that promotion of the participatory approach was an integral part of project design, but it was not captured in the statement of development objectives. 3.27 The relevance of design is rated Substantial. Implementation 3.28 The project was approved on October 31, 2002 and became effective on July 14, 2003. The project effectiveness date was slightly more than five months later than planned, mainly due to longer-than-expected processes for final approval and loan signing. The Mid- Term Review was postponed by a year until January 2007 in response to a request from the Government. In light of the start-up delays, the Government wanted more time for implementation to progress further prior to the mid-term review. The project closed on July 31, 2009, seven months after the original closing date of December 31, 2008. The closing date was extended to complete cropping activities that overlapped two calendar years. PLANNED VERSUS ACTUAL DISBURSEMENTS 3.29 Total project costs were US$44.74 million equivalent, slightly less than the US$ 44.86 million appraisal estimate. The project was financed by an IBRD loan of US$34 million equivalent, corresponding to 77 percent of the estimated project costs. During implementation 24.5 percent of the original loan amount in Euros was cancelled due to a 25 depreciation of the Tunisian dinar. At project closure US$34.48 million equivalent had been disbursed,9 which represented 77 percent of total project costs in U.S. dollar terms. 3.30 At appraisal the government was expected to contribute US$6.85 million equivalent, corresponding to 15 percent of estimated project costs. The actual government contribution at project closure was US$7.7 million, 17 percent of project costs. Beneficiary communities were expected to contribute US$4.01 million, 9 percent of the estimated project costs. The community contribution at project closure was US$2.95 million, 7 percent of total project cost. 3.31 Table 6 summarizes the total project costs as planned, versus actual disbursements against each of the components. 3.32 The over-disbursement for Component 1 resulted from the higher than expected demand for institutional support (training workshops) to beneficiaries and partners, and from technical assistance that was more costly than predicted. 3.33 The under-disbursement for Component 2 — Pilot Operation Execution — was due to the late launch of community development plans under pilot operations and the incomplete execution of the micro-enterprise subcomponent. 9 The loan was made in Euro terms. The total commitment was Euro 34.7 million, Euro million was disbursed and Euro 8.5 million was cancelled. The final loan disbursement in US$ equivalent appears to be higher than the total US$ equivalent loan commitment because of changes in the exchange rate over the project implementation period. 26 Table 6. Project Cost by Component (in USD million equivalents) Appraisal Estimate Actual Amount Percentage of Components (USD millions) (USD millions) Appraisal 1. Institutional Strengthening for 3.01 3.52 117.0% ODESYPANO and Partners 2. Implementation of Pilot 3.25 2.44 75.0% Operations 3. Agricultural and Livestock 3.14 2.39 76.0% Development 4. Sustainable Natural Resources 14.85 16.41 110.5% Management 5. Improvement of Basic Rural 13.79 19.99 145.0% Infrastructure Total baseline costs 38.04 44.74 117.6% Physical Contingencies 0.85 0.00 Price Contingencies 5.97 0.00 Total Project Costs 44.86 44.74 Front-end fee IBRD 0.00 0.39 Total Financing Required 44.86 45.13 100.6% Source: World Bank 2010 3.34 The under-disbursement of Component 3 — Support for Agricultural and Pastoral Production — was due to several factors: (i) the number of trainings provided to the Agricultural Committees and specialists was less than what was planned; (ii) the execution of the milk livestock advisory approach was implemented through the ODESYPANO, instead of the Office of Livestock and Pasture, with associated cost savings; and (iii) the number of realized water springs was less than planned due to the weak water potential of these sources (insufficient flow and depletion in summer periods). 3.35 The over-disbursement for Component 4 — Support for Protection/Management of Natural Resources — was the result of the high demand in the community development plans for pastoral, forestry-pastoral, and agroforestry actions. However, it is also reported that the use of local contractors for the completion of some component activities subsequently reduced costs. 3.36 The over-disbursement of Component 5 — Rural Infrastructure Improvement — was caused by higher than anticipated demands expressed in the community development plans, especially for potable water and road construction/rehabilitation. IMPLEMENTATION EXPERIENCE 3.37 Project implementation was affected by the following challenges. 3.38 Start up Delays. Project effectiveness was delayed by 5 months because the loan signing process took longer than anticipated. Technical assistance to strengthen the 27 implementing agency was delayed by two years because of difficulties in identifying a qualified provider capable of providing support on a large scale. These delays were eventually made up during the course of implementation. 3.39 Lack of adequate access to micro-credit to promote income-generating micro- projects. The project originally intended to support income-generating activities through two types of investments: micro-projects (up to 10,000 TD) and microenterprises (from 10,000 TD up to 50,000 TD). Supervision reports indicate that micro-credit from the Tunisian Bank of Solidarity did not materialize during implementation as planned. Consequently support for micro-enterprises was dropped and the number of micro-projects developed was constrained. Micro-project development was also hampered by the lack of sufficient technical expertise. MONITORING AND EVALUATION IMPLEMENTATION AND USE 3.40 The project’s completion report notes that data was collected on a regular basis, but there were several weaknesses in M&E implementation. The implementing agency found that some of the key performance indicators were difficult to measure and reported on national statistics instead, but no control groups were established to assess the projects attribution. Targets were not established until three years into implementation. A baseline survey was not carried out. Instead, at project closure ODESYPANO recreated a baseline scenario by drawing on data included in the participatory development plans. 3.41 Technical assistance was provided to improve ODESYPANO’s methodology for defining indicators and methods for data collection and developing baselines. But it is not clear to what extent this was applied to monitoring the project. A methodology to conduct household surveys to assess impacts of agricultural and non-agricultural activities was tested with a small sample (380 households drawn from 37 participatory development plans), but this was not used to assess project impacts. It was also anticipated that ODESYPANO’s GIS system would be upgraded as part of technical assistance to strengthen its capacity to monitor environmental impacts and supervise safeguards, but this did not occur. Beneficiaries participated in monitoring the implementation of participatory development plans. 3.42 The MIS system was useful in tracking completion of activities but not in assessing development outcomes. Reporting in Bank supervision reports on progress towards meeting project objectives was delayed until 2007. Much of the data recorded in Bank supervision reports and the project completion report is of limited use in assessing actual results. There were no indicators to measure impacts on soil loss, siltation or water recharge and no assessment was made of changes in community capacity. While communities were involved in monitoring the implementation of their participatory development plans, beneficiary input into the project completion report was limited. There was no stakeholder survey or focus groups to systematically capture their input. SAFEGUARDS COMPLIANCE 3.43 At appraisal the project was classified as category B under the World Bank’s Operational Policy 4.01 (Environmental Assessment) since only limited potential environmental impacts were anticipated related to the construction of rural infrastructure 28 facilities (roads, small mechanical soil and water conservation structures, and the small-scale irrigation perimeters). An Environmental Management Framework Plan was prepared that required the implementing agency to screen subprojects for environmental and social triggers and for subsequent review and clearance by the Tunisian Environmental Protection Agency and followed by appropriate mitigation measures. No other safeguard policies were triggered. 3.44 Project files indicate that during implementation the Bank fell short of ensuring adequate application of the safeguards measures put in place at appraisal. The project’s completion report notes that although the project did not trigger the World Bank’s operational policy for involuntary resettlement at appraisal, given that the subprojects were demand-driven and had not been identified at appraisal, the Bank supervision team should have reassessed social safeguards during the project’s Mid-Term Review. This did not happen. None of the Bank’s supervision missions included a safeguards specialist. An assessment of safeguards implementation status was not carried out until 15 months prior to project completion as part of a safeguards thematic review mission conducted by the World Bank’s Middle East and North Africa region safeguard team. 3.45 The safeguard review found that the Environmental Management Framework Plan was only partially implemented. The implementing agency received training on environmental impact assessments but it did not hire staff dedicated to the supervision of safeguards implementation as had been agreed during project preparation and did not adequately screen subprojects. Supervision documents indicate that the implementing agency did not conduct subproject environmental assessments because it was informed by the Tunisian Environmental Protection Agency that screened sub-projects were exempt from environmental assessment according to Tunisian Environmental Impact Assessment legislation. 3.46 The safeguard review also found that some sub-projects had resulted in the temporary occupation of private properties. The implementing agency obtained verbal consent from the affected landowners during the community development planning process, but this did not comply with the World Bank safeguards requirement for written authorization. The project completion report indicates that the implementing agency team was not aware that written authorizations were required for temporary occupations. The implementing agency’s view, reported in project supervision reports and reiterated to the IEG mission, was that the social impacts generated by temporary occupations were minimal and resolved in a mutually satisfactory manner. 3.47 Project supervision documents indicate that following the safeguard review the implementing agency reinstated environmental screening for the remaining sub-projects. However, the review occurred too late in the implementation cycle to resolve the lack of written documentation of landholders’ consent for temporary occupation of their land. No negative effects from inadequate safeguard compliance were detected by subsequent supervision missions or the safeguards diagnostic review carried out in preparation of a follow-on operation. Funds have also been allocated under the follow-on operation to improve the implementing agency’s capacity to monitor implementation the Bank’s safeguard policies. 29 Achievement of the Objectives OBJECTIVE 1: ENHANCE HOUSEHOLD INCOMES AND SOCIO-ECONOMIC CONDITIONS THROUGH THE IMPROVEMENT AND DIVERSIFICATION OF THE AGRICULTURAL AND PASTORAL PRODUCTION SYSTEMS AND THE PROMOTION OF OFF-FARM INCOME- GENERATING ACTIVITIES 3.48 The project aimed to increase household incomes primarily through investments to improve and diversify agricultural and pastoral production systems. Socio-economic conditions were to be enhanced by improving beneficiaries' access to basic rural infrastructure, specifically water supply systems and roads. Two of the pilots carried out under the project also contributed to this objective: a land consolidation pilot to address one of the key constraints to agricultural development in the region and an alternative income generation pilot to promote off-farm income. OUTPUTS 3.49 The project succeeded in meeting many of its output targets for agriculture and livestock production and rural infrastructure activities. 3.50 Agriculture and pastoral production activities comprised the provision of agricultural advisory services, small-scale irrigation and livestock counseling (improved feed, animal husbandry, genetic improvements). 3.51 The project completion report shows that progress was made in terms of increased yields, diversification of agricultural production systems, and land use rationalization. Agriculture- related activities resulted in an increase in cultivated areas, better yields for crops, meat and milk, and a slight shift in cropping patterns towards higher value production, in horticulture and arboriculture.  Yields for main crops improved. Wheat yields increased 14 qx/ha in the first year of project to 19qx/ha in the final year and olive trees from 13 qx/ha to 24qx/ha — both in line with the appraisal targets of 20 qx/ha for wheat and 19qx/ha for olive trees.  Cultivated areas in the project areas increased from 17 percent to 23 percent for fodder crops (target 25 percent) and from 0.8 percent to 2.0 percent for market gardening (no target). The percent of fallow land in the project areas fell from 21.7 percent to 11 percent in the final year of the project. 3.52 The reliability of this data, however, is questionable. In the course of IEG’s desk review of the project completion report a number of concerns were raised over the methodology used to collect this data. There were gaps in the baseline values, sample sizes were not reported, and control groups were not established. The reviewer also questioned the validity of the data as there was no assessment independent of the implementing agency. The IEG mission did not find any additional information to resolve these concerns. 3.53 Rural Infrastructure. 906 individual water tanks (176 percent of the appraisal target of 515 tanks) and eight potable water connections were completed (133 percent of appraisal target). The project constructed 272 km of roads and rehabilitated 599 km of roads (compared to the targets of 232 Km and 532 km). 30 Table 7. Agriculture Production System Results Year/Quantification INDICATOR UNIT 2003 2004 2005 2006 2007 2008 2009 Land and Crop Occupation Cereals % 45.9 37.3 37.1 40.1 Leguminous crops % 3.4 4.7 4.8 5.3 Forage % 16.6 22.1 22.4 23.4 Arboriculture % 11.6 16.1 17.4 17.9 plantations Vegetable crops % 0.8 2.1 2 2.1 Fallow % 21.7 17.7 16.2 11.1 Main crop yields Wheat Qx/ha 14.2 17.9 17.4 13.3 18.9 Barley Qx/ha 13.9 17.1 16.5 12.7 21.3 Beans Qx/ha 6.9 9.0 10.9 8.5 10.6 Olives Qx/ha 12.6 18.5 21.9 18.2 23.6 Meat yields Pure race bovines Kg/head/year 277 348 370 355 375 Mixed race bovines Kg/head/year 178 210 231 215 235 Local race bovines Kg/head/year 102 123 125 117 127 Sheep Kg/head/year 30 37 37 31 35 Goats Kg/head/year 22 27 27 22 25 Milk yields Pure race bovines L/head/an 2,650 3,326 3,873 3,450 3,911 Mixed race bovines L/head/an 1,060 1,350 1,468 1,260 1,544 Local race bovines L/head/an 285 352 414 380 468 Source: Project Completion Report 3.54 Land consolidation pilot. Six land consolidation operations were completed resulting in the consolidation of 5,346 hectares (99 percent of target of 5,400 ha). 31 km of roads were built to improve access to the consolidated plots (representing 58 percent the appraisal target and 74 percent of the target identified in the participatory development plans). An evaluation of the pilot showed a reduction in the fragmentation of plots and an increase in the average size of farm plot, which in turn reduced the costs of mechanization and transporting inputs to farm and product to markets. Conflicts over land rights were reduced in the areas were consolidation was successful.10 10 In a few areas consolidation efforts were stalled because conflicts between farmers could not be resolved. 31 3.55 Alternative income-generation pilot. Training was provided for 1,863 persons in the following topics: small-scale processing of agricultural produce (1,058 persons), crafts (728 persons) and essential oils extraction (77 persons). The number of people who received training was more than four times the appraisal target of 440. But only 685 beneficiaries (37 percent of the people trained) were successful in obtaining credit and established a micro-project. The project completion report estimated the average revenue per micro-project as TD 2,000 per year. Supervision documents indicate that only 40 percent of the enterprises were operational at project closure and the others were in decline. IEG was unable to determine the number of micro-projects that continued to operate at the time of the assessment mission. OUTCOMES 3.56 The project completion report noted that average household income in the region as a whole increased from TD 2,050 in 2003 to TD 3,784 in 2009 in constant terms.11 The implementing agency found it difficult to assess household incomes, so national statistics for the region were reported, but no control groups were established to assess the extent to which the change in income could be attributed to project activities as opposed to other factors. National statistics also showed that unemployment rates in the region dropped from 19 percent in 2003 to 16 percent in 2009,12 but the same attribution issues apply. 3.57 The project completion report shows that access to potable water increased from 69 percent to 81 percent over the project time period (in line with the appraisal target of 80 percent). Access to roads increased from 56 percent to 81 percent over the course of implementation (no target was established for this indicator). With respect to attribution, the project team informed IEG that the project was the only government intervention supporting rural roads and water supply services in the project area during this time. Hence it is reasonable to assume that the project activities were a contributing factor to the reported increase in access. The project appraisal document presumed improving access to rural infrastructure would contribute to the objective of enhancing the socio-economic conditions of the population in the project areas, but outcome-level results from increased access to infrastructure were not measured. 3.58 In light of the lack of outcome level evidence that can be attributed to the project, achievement of this objective is rated modest. OBJECTIVE 2: ENSURE SUSTAINABLE MANAGEMENT OF THE NATURAL RESOURCES THROUGH THE IMPROVEMENT AND DIVERSIFICATION OF THE AGRICULTURAL AND PASTORAL PRODUCTION SYSTEMS AND THE PROMOTION OF OFF-FARM INCOME- GENERATING ACTIVITIES 3.59 This objective was primarily pursued by promoting planting of tree crops (olives, almonds), pasture and rangeland improvements, and soil and water conservation works. A 11 Annual increases in income reported were TD 2,050 in 2003, TD 3,900 in 20007, TD 3,280 in 2008, and TD 3,784 in 2009. The decline in income in 2008 was attributed to a shortfall in rain during that cropping season. These figures reflect national statistics for the North West Region as a whole. 12 The target was for unemployment rates to drop to 10 percent by project closure. The project completion report notes that this was overoptimistic given the fact that the national level of unemployment was 13.9 percent and historically the rates in the Northwest region have been higher than the national average. 32 pilot to test the use of the participatory approach in communities adjacent to forests was intended to promote greater engagement of communities in forest management activities. OUTPUTS 3.60 The project exceeded its target in terms of the area treated by soil and water conservation works,13 improved pasture,14 and planting of tree crops (olives, almonds). 54,880 hectares were treated compared to the 48,000 hectare appraisal target. Consolidation of farm parcels under the land consolidation pilot also contributed to this objective, albeit on a small scale, since it allowed farmers who benefitted from land consolidation efforts to change the direction of plowing on slopes to reduce erosion. However, this result was not quantified. The project also intended to contribute to this objective by developing participatory forest management plans under the forest pilot, but implementation was not carried out by project closure. OUTCOMES 3.61 There is limited evidence to indicate the extent to which project activities are contributing to sustainable natural resource management. The project completion report indicates that vegetation and forest cover in the project areas grew from 32 percent to 38 percent over the project period (a 19 percent increase in vegetation cover compared to the appraisal target of 20 percent). The project did not report on survival rates of the trees planted and control groups were not established to determine how much of the vegetation increase was due to project activities. Estimates of changes in soil loss, dam siltation rates, and infiltration or water recharge were not assessed. There is also insufficient information to determine community management practices or the incentives that they have to maintain the project’s investments in soil and water conservation works, rangeland improvements, or tree crops. 3.62 Achievement of this objective is rated modest. Efficiency 3.63 At appraisal the economic rate of return for the project as a whole was estimated to be 17 percent over 20 years. The cost stream at appraisal included agro-forestry and pasture improvement investments, 80 percent of ODESYPANO capacity development costs and investments made over the project implementation period as well as replacement, operating and maintenance costs. The benefit stream included quantifiable benefits derived from additional agricultural production, rural infrastructure development and off-farm activities. Yield increases by crop were applied to the project areas based on an assumed adoption rate of 25 percent per annum, reaching 100 percent after four years. Extension coverage was estimated at 50 percent of the sown area in the first year and 100 percent in the second. The ERR calculation did not take into account benefits arising from the improved access to rural 13 Soil and water conservation works comprised stone sills to protect tree crops and stone walls and plantations to control gully erosion against water runoff. 14 Pasture improvements included including establishment of resting areas, reseeding and fertilizing. The project completion report states that before the project these measures were not practiced in the project areas. 33 roads and water supply systems, increased vegetation and forest cover, provision of environmental training, which the completion report assumes to have resulted in better protection and management of natural resources, institutional strengthening of ODESYPANO, and strengthening of beneficiary grass roots organizations. 3.64 At closure the economic rate of return was recalculated at 27 percent over 20 years, using the same methodology and approach used by calculations made at appraisal. The project’s completion report attributed the increased rate of return to improved technology adoption, higher value added crops such as olives, fruit and vegetables, and improved infrastructure allowing for better commercialization. Sensitivity analysis that took into account cost overruns and profitability reductions of 20 percent showed that ERR sensitivity was modest. However, it is likely that the economic rate of return is overoptimistic. Adoption rates were based on estimates as opposed to actual achievements and the project only monitored increased in production. It did not provide evidence that commercialization of products increased. In addition, it is not possible to estimate accurate incremental economic benefits without control groups and these were not established. 3.65 In terms of efficiency in the management of project resources, the final approval and loan signing took five months longer than expected and the project also experienced start-up delays due to the complexity of organizing and mobilizing the delivery mechanism for the large-scale technical assistance to strengthen the ODESYPANO organizational structure at the project start. But these delays were made up during the course of implementation. The project achieved efficiency gains in the preparation of community development plans by moving to larger planning units than had been used in the past. Previously community development plans had been prepared by individual villages. During implementation neighboring villages within the next largest local administrative unit were clustered together. The project completion report noted that the use of the larger planning unit allowed for gains in time and money, while maintaining the original project targets in terms of territorial coverage (500,000 ha) and population coverage (260,000 inhabitants). 3.66 Overall, efficiency is rated modest. Ratings OUTCOME 3.67 The overall outcome of the Northwest Mountains and Forest Areas Project is rated moderately unsatisfactory. The project objectives were substantially relevant to Bank strategies and Government policies in effect when the project was approved and remain relevant to current strategies. Project design was substantially relevant to achieving its objectives. However, achievement of both project objectives was modest due to lack of outcome level evidence and attribution questions. Efficiency was also modest. RISK TO DEVELOPMENT OUTCOME 3.68 Risk to development outcome is assessed from two aspects — the likelihood that investments implemented under the project will be maintained and the risks to the durability of the integrated participatory approach mechanism. 34 3.69 Project supervision reports indicate that operation and maintenance arrangements are in place and functioning for most infrastructure investments. The project was particularly proactive in ensuring that maintenance arrangements would be in place for roads. Under the prior phase of the project, sustainability of roads had been a problem due to a lack of clear responsibility for maintaining non-classified rural roads. In light of that experience all roads constructed or rehabilitated under the project received standard surfacing which upgraded them to the level of “classified” roads, thereby placing them directly under the Ministry of Public Works responsibility for their maintenance. Roads and irrigation systems viewed by the IEG mission were in good operating condition, although the mission only visited a small non random sample. 3.70 Sustainability of natural resources management practices is less certain. The project completion report noted that over the course of the project communities had demonstrated an increased interest and understanding of how to establish and regularly apply sustainable natural resource management practices, including better forms of production, cropping patterns, and diversification favoring natural resource conservation and better use of available resources. But the communities’ incentive to sustain these practices in unclear. Project documentation and IEG interviews did not shed light on this issue. 3.71 The durability of the integrated participatory approach mechanism depends on long- term institutional support and the willingness and ability of the different actors to continue participating. The government has expressed its commitment to the model through its decision to make ODESYPANO a permanent entity15 and its request for a follow-on phase of the project to consolidate the project’s gains. However, questions remain about ODESYPANO’s financial sustainability in the absence of a World Bank project. Since the agency’s inception its budget has been financed jointly through the government and World Bank project funds. The follow-on project was approved in late 2010 shortly before Tunisia’s revolution. This experienced a number of start-up delays related to implementation challenges that emerged in aftermath of the revolution but at the time of the writing of this review implementation had begun to move forward. 3.72 The continued engagement of ODESYPANO has minimized disruptions to the IPA process in the project areas following revolution. The approval of participatory development plans was disrupted due to the dissolution of regional and local government councils. At the community level the formally recognized agriculture development associations across the country collapsed. Communities viewed the leaders of many of these groups as an arm of the government rather than legitimate representatives of the community and rejected their leadership following the revolution. In some cases tensions were such that these individuals were forced to leave their communities. IEG interviews indicated that ODESYPANO played an important role in overcoming this leadership crisis by helping communities to conduct new elections and revalidate the priorities that had been included in the participatory 15 The project was prepared with the expectation that ODESYPANO’s role in promoting the IPA would be phased out by the end of the project. Responsibilities were expected to be transferred to the CRDA. As IEG’s assessment of the prior project indicates, the CRDAs were not ready to take on this role. In 2008 the government commissioned a study on the future of local development in the Northwest after the project. The study concluded that further large-scale investments were still needed in the region. It found that ODESYPANO had a comparative advantage implementing the approach in the region and recommended that it continue in this role. 35 development plans prepared prior to the revolution, including preparing new plans in cases when old plans had been rejected. 3.73 Despite ODESYPANOs effort to restore legitimacy to community organizations, a key risk to the development outcomes of the project is that the types of associative structures that currently exist in Tunisia have a number of limitations that hinders their utility as a vehicle to formally represent the interests of the community as a whole. The project has worked through two types of community organizations: Community Development Committees (CDs) and Agriculture Development Groups (GDAs). CDs are informal entities created by the project that lack legitimacy to engage directly with government agencies on their own because they do not have an official legal status. The second category, the Agriculture Development Group, is a legally recognized associative structure. The project supported the transformation of many CDs to GDAs to ensure greater sustainability but these do not represent all segments of the population. The GDA status was created for the purpose of managing natural resources in a common area (similar to water-user associations) and only those with agricultural land are eligible for membership. The landless and community members who are not agriculturalists cannot be a member of a GDA. A second limitation of this type of association is that they are prohibited from engaging in commercial activities and generating profits, thereby limiting their financial sustainability and capacity for self- management.16 3.74 The risk to development outcome is assessed as moderate. BANK PERFORMANCE Quality at Entry 3.75 Project preparation drew on the experience of prior Bank-funded operations in the country with similar objectives and on technical background papers financed by a project preparation grant. The project’s design was complex in that it financed multiple activities through 15 subcomponents and involved partnerships with multiple entities. The Bank team was proactive in recognizing that implementation of such a complex design would require strong management, operational, and coordination capacities and invested in training of ODESYPANO staff prior to implementation, in development of methodological tools needed to support the participatory approach, and in strengthening ODESYPANOs logistical capacity. The project appraisal document appropriately identified many project risks, but there were some gaps. The project team did not foresee the risk that the CRDA would not be prepared assume ODESYPANO’s role in facilitating the preparation of community development plans by project closure so that no mitigation measures were prepared. The difficulties in developing income-generating micro projects were underestimated — specifically, the challenge in accessing credit and the availability of competent consultants to promote micro-project development activities. The project design was also over-optimistic with respect to the time required for effectiveness and procuring technical assistance to further develop ODESYPANO’s capacity during the course of implementation. In addition, there were weaknesses in the design of the project’s monitoring and evaluation system. 16 Initially the legislation governing GDAs allowed them to engage in commercial activities such as selling their member’s products. In 2004 the law was changed prohibiting them from engaging in any commercial activities. 36 3.76 Overall the Bank’s performance in ensuring quality at entry is rated moderately satisfactory. Quality of Supervision 3.77 Project documentation and IEG interviews indicate that throughout project implementation the Bank team provided adequate financial backstopping and support for resolution of implementation challenges. Supervision missions included field visits to remote areas. Supervision reports indicate that the field visits contributed to a greater understanding of implementation challenges and in developing adequate solutions. However, documents in the project files also point to a number of weaknesses in supervision. The frequency of supervision missions was below the recommended average of two missions per year17 and supervision teams lacked certain specialized expertise requested by the implementing agency. Supervision missions were fielded by teams of 3 to 4 members. Project files indicate that although some team members were experienced with environmental and social safeguards, and monitoring and evaluation, in addition to their primary area of expertise, none of the missions included a specialist dedicated to these topics. Supervision reports also reveal weak monitoring of progress to development outcomes during the first half of the implementation period. The project team’s assessment of progress towards meeting development outcomes was based on the high levels of community participation and number of project investment activities carried out and not on the actual achievement of results. Management comments during this period reflect concerns with the lack of targets for many key performance indicators, the team’s failure to report and analyze data on the project development indicators and a lack of clarity on the meaning of some indicators. Finally, supervision of environmental and social safeguards was inadequate. An assessment of safeguards implementation status was not carried until April 2008, 15 months prior to project completion, when a safeguards thematic review was conducted for several projects in the country. The review found that the Environmental Management Framework Plan prepared at project appraisal had not been fully implemented and that some sub-projects had resulted in the temporary occupation of private properties. The implementing agency had obtained oral consent from the affected landowners but this did not strictly comply with the World Bank’s safeguards procedures that require written authorization. Moreover, the project’s completion report noted that although the project did not trigger the World Bank operational policy for involuntary resettlement at appraisal, the Bank supervision team should have reassessed social safeguards during the project’s Mid-Term Review and provided support in documenting landowners’ consent and in preparing for appropriate compensation where warranted. This did not happen. 3.78 The Bank’s supervision performance is rated moderately unsatisfactory. 3.79 The overall Bank performance is rated moderately unsatisfactory. This is in accordance with the IEG/OPCS harmonized guidelines that specify that when the two aspects of Bank performance are split the overall performance rating follows the outcome rating. 17 During the project implementation period, 2003 to 2009, a total of nine supervision missions were carried out — an average of 1.5 per year — and there was a twelve month gap between three missions. 37 BORROWER PERFORMANCE Government Performance 3.80 The government demonstrated its commitment to the project by disbursing counterpart funds on time throughout implementation and contributing more than originally planned. Project files indicate that government coordination committees set up for the project met on a regular basis and functioned as planned. There was some disruption to project implementation due to the turnover of elected officials (part of the normal election cycle) who needed to be brought up to speed on the project’s goals and partnerships and approval requirements. In some instances the incoming officials had different views from their predecessors on how the project should be implemented. The government approved a decree allowing Community Development Committees established by the project to transition to Agriculture Development Groups. However, the legislation governing the status of ADGs was revised in 2004 — restricting them from engaging in commercial activities — which is counterproductive to their financial sustainability. 3.81 Government performance is rated moderately satisfactory. Implementing Agency Performance 3.82 The implementing agency, performed effectively in design and implementation of activities. Financial management performance was satisfactory throughout implementation. Audits were carried out on time and did not reveal any significant issues. Procurement was also satisfactory. However there were shortcomings in monitoring and evaluation and the supervision of environmental and social safeguards. The implementing agency did not hire a staff member dedicated to the supervision of safeguards implementation as had been agreed during project preparation and the Environmental Management Framework Plan was not fully implemented. The implementing agency never recruited a staff member responsible for implementation of environmental safeguards as had been planned during appraisal stage, so that environmental measures in the project design document were only partially implemented. Safeguards are discussed in further detail in paragraphs 3.43 to 3.47. 3.83 Implementing Agency performance is rated moderately satisfactory. 3.84 Taking into account the ratings for government and implementing agency performance, the overall Borrower Performance is rated moderately satisfactory. MONITORING AND EVALUATION 3.85 Design. Performance indicators were identified at appraisal with targets but there were gaps in key output indicators. 3.86 Implementation. Data was collected on a regular basis but some indicators proved difficult to measure. Targets were not established until mid-project and baseline data was not established until the end of the project. No control groups were established to assess the attribution of project activities to reported outcomes. Beneficiaries participated in monitoring the implementation of participatory development plans and had access to feedback 38 mechanisms on project implementation. There was limited beneficiary input into the project’s completion report. 3.87 Utilization. The projects MIS and beneficiary monitoring was useful for implementation but M&E data was of limited use in assessing the projects results. 3.88 Overall, the quality of M&E is rated negligible. 4. Lessons 4.1 The experience of the two projects yields the following lessons: 4.2 Weaknesses in the structures that are used to integrate communities into the development process can undermine securing genuine collaboration of communities over the long term. The current forms of organization available to communities in Tunisia have a number of limitations that hamper their utility as a vehicle to formally represent the interests of the community as a whole. It is important that community organizations have sufficient legal standing to ensure their legitimacy in representing the communities’ priorities to government agencies, that communities view the leaders of such organizations as legitimate, and that all segments of the population are represented. 4.3 Demonstrating the effectiveness of natural resource management interventions requires attention to monitoring and evaluation of actual outcomes. Despite three decades of World Bank support to integrated rural development projects with significant investments in soil and water conservation works, no assessment has been made to determine the actual impacts of these interventions on erosion, soil fertility, groundwater recharge, or dam siltation. To date reporting of natural resources management achievements has been based on assumed impacts. Measuring actual results is not only important for understanding whether the intended development impact has been achieved, but also for assessing the cost- effectiveness of the measures supported and generating information on what needs to be improved. 4.4 Effective promotion of income-generating projects requires more than support for production. Attention to markets and ensuring the availability of credit is also crucial. Under both projects support for alternative income-generating projects was limited to training and technical support for production side issues. The projects had limited success in establishing successful income-generating enterprises. Many of the training participants were unable to access existing sources of credit. Income-generating activities were also hampered by a lack of market studies. 39 References Canesse, Aude-Annabelle. 2010. “Rural “Participation” and its Framework in Tunisia.” Journal of Economic and Social Research 12(1): 63–88. ODESYPANO. 2009. Etude d'évaluation des opérations pilotes de remembrement des terres en sec dans les zones d’intervention de l’ODESYPANO. IEG/OED. 1993. “Performance Audit Report for Tunisia Northwest Rural Development Project.” Report No. 12031. Washington, DC. _______. 2003. “Project Performance Assessment Report Republic of Tunisia for: Agriculture Research and Extension Project; Agricultural Sector Investment Project, Northwest Mountainous Areas Development Project; National Rural Finance Project.” Report no. 26260. Washington, DC. World Bank. 1997. “Project Appraisal Document for the Natural Resources Management Project.” Washington, DC. _______. 2000. “Country Assistance Strategy of the World Bank Group for the Republic of Tunisia.” Report No. 20161-TN. Washington, DC. _______. 2002. “Project Appraisal Document for the Northwest Mountains and Forest Areas Project.” Washington, DC. _______. 2004a. “Implementation and Completion Report for the Natural Resources Management Project.” Washington, DC. _______. 2004b. “Country Assistance for the Republic of Tunisia.” Report No. 28791-TUN. Washington, DC. _______. 2006. “Tunisia Agricultural Policy Review.” Washington, DC. _______. 2010a. “Implementation and Completion Report for the Northwest Mountains and Forest Areas Project.” Washington, DC. _______. 2010b. “Project Appraisal Document for a Second Natural Resources Management Project.” Washington, DC. _______. 2010c. “Project Appraisal Document for a Fourth Northwest Mountains and Forest Areas Project.” Washington, DC. _______. 2010d. “Development Policy Review Towards Innovation-driven Growth.” Washington, DC. _______. 2012a. “Tunisia: Agricultural Finance Study.” Washington, DC. _______. 2012b. “Interim Strategy Note for the Republic of Tunisia for the period of FY13–14.” Report no. 67692-TN. Washington, DC. _______. 2013. “Tunisia in a Changing Climate: Assessment and Actions for Increased Resilience and Development.” Washington, DC. 41 ANNEX A Annex A. Basic Data Sheet NATURAL RESOURCES MANAGEMENT PROJECT (LOAN NO. 4162; TF-25830) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 51.3 35.4 69 Loan amount 26.5 20.9* 79 Government counterpart 16.6 9.32 56 Beneficiary contribution 6.5 4.98 76 Cofinancing 1.7 0.2 12 Cancellation NA 0.95 NA * Data reported in ICR Cumulative Estimated and Actual Disbursements FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 Appraisal estimate (US$M) 0.3 3.3 9.0 19.0 24.5 26.5 -- -- Actual (US$M) 0.84 0.96 1.93 5.44 10.1 15.2 19.07 22.27* Actual as % of appraisal 280 29 21 29 41 57 0 0 Date of final disbursement: June 2003 * Data reported in SAP Project Dates Original Actual Initiating memorandum 11/15/1995 11/15/1995 Board approval 05/13/1997 05/13/1997 Effectiveness 10/15/1997 04/15/1998 Closing date 06/30/2003 06/30/2004 Staff Inputs (staff weeks) No. Staff weeks US$ (‘000) Identification/Preparation Not available Not available Appraisal/Negotiation 152 547,680.76 Supervision 193 695,088.06 ANNEX A 42 Other Project Data Borrower/Executing Agency: Follow-on Operations Operation Credit no. Amount Board date (US$ million) Second Natural Resources Management Project 79210 36.1 06/17/2010 43 ANNEX A Basic Data Sheet NORTHWEST MOUNTAINS AND FOREST AREAS DEVELOPMENT PROJECT (LOAN NO. 7151, TF-26699) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 44.9 45.13 101 Government 6.9 7.70 112 Local contribution 4.0 2.95 74 Loan amount 34.0 34.48 101 Cancellation * * The loan was made in Euro terms. The total commitment was EURO 34.7 million. EURO 8.5 million was cancelled. Cumulative Estimated and Actual Disbursements FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 Appraisal estimate (US$M) 0.8 4.8 10.7 17.2 24.2 31.2 34 34 Actual (US$M) 0 1.5 3.6 7.9 15.6 24.7 32.6 34.1 Actual as % of appraisal 0 31 34 46 65 79 96 100 Date of final disbursement: 2/24/2010 Project Dates Original Actual Initiating memorandum 08/01/2001 06/14/2001 Board approval 10/31/2002 10/31/2002 Signing 12/23/2002 12/23/2002 Effectiveness 07/14/2003 07/14/2003 Closing date 12/31/2008 07/31/2009 Staff Inputs (staff weeks) Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of staff weeks USD thousands (including travel and consultant costs) Lending 64 417.81 Supervision/ICR 63 432.35 ANNEX A 44 Task Team Members Title (at time of appraisal and Responsibility/ Name Unit closure, respectively) Specialty Lending Idah Pswarayi-Riddihough Sr. Natural Res. Management Specialist MNSRE TTL-I Kutlu Somel Economist MNSRE Concepcion del Castillo Senior Social Scientist MNSRE Sarnia Msadek Sr. Financial Management Specialist MNSRE Harry Palmier Institutional Development Specialist AFTRE Douglas Lister Senior Agricultural Economist MNSRE Petros Aklilu Sector Manager Rural Development MNSRE Nadia Gouhier Procurement Analyst MNSRE Hovsep Melkonian Senior Disbursement Officer LOAG1 Hassine Hedda Disbursement Analyst LOAG1 Arbi Ben-Achour Senior Social Scientist MNSRE Alan Rotman Senior Environment Specialist MNSRE Hocine Chalal Environmental Specialist MNSRE Dominique Bichara Senior Counsel LEGOP Maurice Gress Principal Procurement Specialist MNA Marjory-Anne Bromhead Sector Manager, Peer Reviewer ESSD Alexandre Marc Sector Manager, Peer Reviewer ESSD Christopher Ward Lead Oper. Officer, Peer Reviewer MNSRE Michelle Keane Senior Country Officer MNC01 Aloysius Ordu Regional Quality Advisor MNACS Laurent Msellati Sr. Oper. Off. – Procurement Accr. Staff MNSRE Rafika Chaouali Sr. Financial Management Specialist MNACS Issam Abousleiman Senior Financial Officer BCFPS Supervision/ICR Anas Abou El Mikias Sr. Financial Management Specialist MNAFM Jean-Marc Bisson Agricultural Economist Abderrahmane Ben Boubaker Consultant (Spec. Community based Dev. Matters) Slaheddine Ben-Halima Consultant (Procurement) Fatou Fall Social Development Spec. Nadia F-Z. Gouhier Procurement Analyst Gael A. Gregoire Environmental Spec. Abdelghani Inal Consultant (Roads) Moez Makhlouf Consultant (Financial Management) Abdelkrim Oka Consultant (Institutional Development) TTL-II Lucie Tran Huong Giang Operations Officer TTL-III Garry Charlier Senior Operations Officer TTL-IV (ICR only) 45 ANNEX A Other Project Data Borrower/Executing Agency: Follow-on Operations Operation Credit no. Amount Board date (US$ million) Fourth NorthWest Mountains and Forestry PN04 41.6 12/10/2010 Areas ANNEX B 46 Annex B. List of Persons Met World Bank Garry Charlier, Task Manager, Second Natural Resources Management Project, Fourth North East Mountains and Forest Areas Project Moez Makhlouf, Financial Management Systems Consultant Abderrahmane Ben Boubaker, Consultant, Participatory Approach Specialist Government Ali Aydi, Director General, Ministry of Agriculture DG/FIOP Lamia Jemali, Ministry of Agriculture DG/FIOP, Project Coordinator, Second Natural Resources Management Project Sana Smida, Ministry of Agriculture DG/FIOP Abdallah Zekri, Director General of Multilateral Cooperation, Ministry of Investment and International Cooperation Mohamed Lotfi Grad, Ministry of Investment and International Cooperation Nourredine Kabbi, Former Director, Ministry of Planning and International Cooperation Chorkri Walha, CRDA Medenine, Regional Project Coordinator, Second Natural Resources Management Project, former staff of Natural Resources Management Project Mabrouk Mouheddine, Jendouba, Regional Project Coordinator, Second Natural Resources Management Project Rjeibi Abdallah, Regional Director ODESYPANO, Ain Draham Regional Office Attafi Hassen, Division Chief, DARAT Souadki Abdel Krim, Regional Coordinator Fernana Fouzai Abdelhamid, Amimator Hdhiri Beya, Aminator Aloui Kamel, ODESYPANO, Project Coordinator, North West Mountains and Forest Areas Project Chalouati Yones, Chief of Rural Animation Services Ben Abdallah Ridha, ODESYPANO, Livestock Counselor Staff of ODESYPANO Ain Draham Regional Office Staff of ODESYPANO Beja Head Quarters Other Donors Rafaa Marouki, Principal Agriculture Economist, AfDB (former project coordinator, Natural Resources Management Project) Sadok Elamri, Program Specialist UNDP (former Central Project Coordinator, Natural Resourced Management Project) Community Members GDA Barbara, Oued Ghrib Ghazouani Abderrazzak, Head of GDA Oued Gherib 47 ANNEX B Ghazonani Hmida, Beneficiary Oued Gherib Jemai Hassen, Beneficiary Oued Gherib Jemai Abdelmagrol, Beneficiary Oued Gherib Ghazoudi Mustapha, Member of Development Committee Hamdi Samir, Beneficiary in Sidi Ammar M. Cerisier, Beneficiary in Djebba