Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004819 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF-95129, TF-95130 ON A GRANT US$49.2 MILLION EQUIVALENT TO THE Forum for Agricultural Research in Africa (FARA) FOR THE Forum for Agricultural Research in Africa (FARA) Trust Fund June 20, 2019 Agriculture Global Practice Africa Region CURRENCY Currency Unit = USD FISCAL YEAR July 1 - June 30 Regional Vice President: Hafez Ghanem Country Director: Deborah Wetzel Senior Global Practice Director: Jurgen Voegele Practice Manager: Mark Cackler Task Team Leader(s): Azeb Fissha Mekonnen, Abdoulaye Toure ICR Main Contributor: Riikka Rajalahti ABBREVIATIONS AND ACRONYMS AF Additional Financing AFAAS African Forum for Agricultural Advisory Services AfDB African Development Bank AGRA Alliance for a Green Revolution in Africa AAIN African Agribusiness Incubator Network AATF African Agricultural Technology Foundation ANAFE African Network for Agriculture, Agroforestry and Natural Resources Education ARD Agricultural Research and Development ASARECA Association for Agricultural Research in East and Central Africa AUC African Union Commission AUDA African Union Development Agency AWP Activity Work Program BMGF Bill and Melinda Gates Foundation CAADP Comprehensive African Agricultural Development Programme CCARDESA Centre for Coordination of Agricultural Research and Development for Southern Africa CGIAR Consultative Group on International Agricultural Research CGRS Corporate Governance Reform Stream CIDA Canadian International Development Agency CILSS Comité Permanent Inter-Etats De Lutte Contre la Sécheresse dans le Sahel/Permanent Interstates Committee for Drought Control in the Sahel CIRDES Centre international de recherche-développement sur l'elevage en zone subhumide CoP Community of Practice CORAF/WECARD West and Central African Council for Agricultural Research and Development CRP CGIAR Research Programs DfID UK’s Department for International Development DONATA Dissemination of New Agricultural Technologies in Africa EC European Commission FAAP Framework for African Agricultural Productivity FANRPAN The Food, Agriculture and Natural Resources Policy Analysis Network FARA Forum for Agricultural Research in Africa IAR4D Integrated Agricultural Research for Development ICART Implementation and coordination of Agricultural Research and Training – project ICIPE International Centre of Insect Physiology and Ecology IFR un-audited Interim Financial Report IPF Investment Project Financing ISR Implementation Status and Results Report ITC International Trypanotolerance Centre (The Gambia) KPI Key Performance Indicators M&E Monitoring and Evaluation MDG Millennium Development Goal MDTF Multi-Donor Trust Fund MTOP Medium-Term Operational Plans NARS National Agricultural Research System NASRO North African Sub-Regional Organization NEPAD New Partnership for Africa’s Development NGO Non-Governmental Organization NORAD Norwegian Agency for Development Cooperation NPCA NEPAD Planning and Coordinating Agency NSF Networking Support Functions OM Operational Manual PAEPARD The Platform for African European Partnership on Agricultural Research for Development PAFFO Pan-African Farmers Forum PanAAC Pan-African Agribusiness and Agro-Industry Consortium PANGOC Pan Africa NGO Consortium PARI Program of Accompanying Research for Agricultural Innovation PDO Project Development Objective PMP Performance Monitoring Plan PRSP Poverty Reduction Strategy Paper (e)RAILS Regional Agricultural Information and Learning System REC Regional Economic Center RUFORUM Regional Universities Forum for Capacity Building in Agriculture RVP Regional Vice Presidency, the World Bank S3A Science Agenda for Agriculture in Africa S4AC Science for Agriculture Consortium SABIMA Strengthening Capacity for Safe Biotechnology Management in Sub-Saharan Africa SADC-MAPP SADC Multi-country Agricultural Productivity Programme SCARDA Strengthening Agricultural Research in Africa SISTA Supporting Implementation of a Science-Led and Climate-Relevant Agricultural Transformation in Africa SP Strategic Priority SRO Sub-Regional Organization SSA CP Sub-Saharan Africa Challenge Programme STI Science Technology and Innovation TEAM Africa Tertiary Education for Agriculture Mechanism for Africa TF Trust Fund TIGA Ex-ante technology assessment for inclusive poverty reduction and sustainable productivity growth in agriculture UNIBrain Universities, Business and Research in Agricultural Innovation USAID The United States Agency for International Development WB The World Bank TABLE OF CONTENTS DATA SHEET ............................................................................................................................................ 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES...................................................................... 6 A. CONTEXT AT APPRAISAL ............................................................................................................. 6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION................................................................. 10 II. OUTCOME ......................................................................................................................................... 13 A. RELEVANCE OF PDOs ................................................................................................................ 13 B. ACHIEVEMENT OF PDOs (EFFICACY).......................................................................................... 14 C. EFFICIENCY................................................................................................................................ 18 D. JUSTIFICATION OF OVERALL OUTCOME RATING....................................................................... 18 E. OTHER OUTCOMES AND IMPACTS (IF ANY)............................................................................... 19 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME .............................................. 20 A. KEY FACTORS DURING PREPARATION....................................................................................... 20 B. KEY FACTORS DURING IMPLEMENTATION................................................................................ 22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ............... 24 A. QUALITY OF MONITORING AND EVALUATION (M&E) .............................................................. 24 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE....................................................... 26 C. BANK PERFORMANCE ............................................................................................................... 26 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................... 28 V. LESSONS AND RECOMMENDATIONS............................................................................................. 29 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS .......................................................................... 33 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION........................................ 52 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................................... 54 ANNEX 4. EFFICIENCY ANALYSIS............................................................................................................ 55 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ................. 58 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................................... 59 ANNEX 7. FARA’S BACKGROUND, MANDATE AND PROGRAMS OVER THE YEARS ................................ 60 ANNEX 8. OTHER CHANGES .................................................................................................................. 61 ANNEX 9. ADDITIONAL INFORMATION ON ACHIEVEMENTS ................................................................. 65 ANNEX 10. MONITORING & EVALUATION – SUPPLEMENTAL INFORMATION...................................... 70 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Forum for Agricultural Research in Africa (FARA) Trust P112684 Fund Country Financing Instrument Western Africa Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Forum for Agricultural Research in Africa (FARA) Forum for Agricultural Research in Africa Project Development Objective (PDO) Original PDO To align African agricultural institutions at the national, regional and continental levels with CAADP Pillar IV FAAPPrinciplesforeffective research, extension, and training and education. Revised PDO To strengthen Africa’s capacity for agricultural innovation with purposefully determined outcomes, creating capacityforinnovationand enabling environments for implementation. Page 1 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 47,807,000 47,807,000 47,807,000 TF-95129 1,902,320 1,491,365 1,491,365 TF-95130 Total 49,709,320 49,298,365 49,298,365 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 49,709,320 49,298,365 49,298,365 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing A 04-Dec-2008 04-Sep-2009 01-Feb-2012 31-Dec-2013 31-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 28-Jun-2012 20.54 Change in Results Framework Reallocation between Disbursement Categories 31-Dec-2013 31.97 Additional Financing Change in Project Development Objectives Change in Results Framework Change in Loan Closing Date(s) Change of EA category 18-Oct-2018 46.81 Additional Financing KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest Page 2 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 15-Dec-2009 Satisfactory Satisfactory 3.44 02 29-Jun-2010 Satisfactory Satisfactory 6.27 03 21-Apr-2011 Satisfactory Satisfactory 11.84 04 03-Jan-2012 Satisfactory Satisfactory 17.19 05 30-Apr-2012 Satisfactory Satisfactory 18.58 06 11-Jul-2012 Satisfactory Satisfactory 20.54 07 05-Feb-2013 Satisfactory Satisfactory 24.17 08 30-Sep-2013 Satisfactory Satisfactory 30.65 09 14-Nov-2013 Satisfactory Satisfactory 31.09 10 18-Jun-2014 Satisfactory Satisfactory 37.52 11 17-Dec-2014 Moderately Satisfactory Moderately Satisfactory 40.57 12 28-May-2015 Moderately Satisfactory Moderately Satisfactory 40.57 13 09-Dec-2015 Moderately Satisfactory Moderately Satisfactory 42.36 14 24-Jun-2016 Moderately Satisfactory Moderately Satisfactory 44.08 15 29-Dec-2016 Moderately Satisfactory Moderately Unsatisfactory 44.53 16 21-Jun-2017 Moderately Satisfactory Moderately Unsatisfactory 45.56 17 18-Oct-2017 Moderately Satisfactory Moderately Satisfactory 46.81 18 07-Mar-2018 Moderately Satisfactory Moderately Satisfactory 46.81 19 23-Apr-2018 Moderately Unsatisfactory Moderately Satisfactory 46.81 Page 3 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 80 Agricultural Extension, Research, and Other Support 80 Activities Education 20 Adult, Basic and Continuing Education 5 Tertiary Education 10 Workforce Development and Vocational Education 5 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Finance 25 Finance for Development 25 Agriculture Finance 25 Urban and Rural Development 75 Rural Development 75 Rural Markets 25 Rural Infrastructure and service delivery 50 ADM STAFF Role At Approval At ICR Regional Vice President: Obiageli Katryn Ezekwesili Hafez M. H. Ghanem Country Director: Mark D. Tomlinson Deborah L. Wetzel Senior Global Practice Director: Jamal Saghir Juergen Voegele Practice Manager: Karen Mcconnell Brooks Mark E. Cackler Abdoulaye Toure, Azeb Fissha Task Team Leader(s): David J. Nielson Mekonnen ICR Contributing Author: Riikka Rajalahti Page 4 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Page 5 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Improvements in agricultural productivity and growth hold a key to poverty reduction in Africa. Agricultural growth has a significant impact on the rest of the economy. A frica’s leaders see agriculture as an engine for poverty reduction and overall economic development. In 2003, the African Union’s New Partnerships for African Development (AU -NEPAD) launched the Comprehensive African Agriculture Development Program (CAADP) that describes African leaders’ collective vision for how to reach a goal of 6 percent per annum growth for the sector. Subsequently, CAADP Pillar IV 1 was launched in 2006 as a strategy to support Africa’s agricultural research, technology dissemination and adoption efforts. Continental coordination of agricultural research & innovation was considered necessary for successful implementation of the CAADP Pillar IV. The AU Commission and AU-NEPAD gave the Forum for Agricultural Research in Africa (FARA) the mandate to serve as the lead institution for coordinating CAADP Pillar IV. FARA was established in 1997 by African regional and science stakeholders and development agencies to address the continent’s fragmented agricultural research systems. Gradually, FARA became a continent-wide umbrella organization that formed coalitions of major stakeholders in agricultural research and development (ARD) in Africa. For further background on FARA, its mandate and programs, see Annex 7. The implementation of CAAP Pillar IV was guided by the Framework for African Agricultural Productivity (FAAP) that was led by FARA and developed (in 2006) in consultation with Africa’s agricultural stakeholders and their development partners (DP). The Framework called for: reforms to agricultural institutions and services; increases in the scale and scope of the investments in agricultural productivity; and aligned and coordinated financial support. A harmonized approach to strengthen Africa’s capacity for agricultural productivity and innovation was pursued to implement CAADP-FAAP. The food price crisis of 2007-2009 brought heightened attention to the need to improve agricultural productivity. This gave rise to the mobilization of increased resources among development partner for ARD in Africa. As a continental coordination platform - FARA became one of the beneficiaries of these resources. The World Bank (WB) and a group of DPs (DfID, CIDA, Netherlands, and the European Commission) proposed the establishment of a recipient executed Multi Donor Trust Fund (MDTF) at the WB. The MDTF was expected to facilitate joint-financing, harmonization of approaches, and assist institutional development of FARA to better address coordination of African ARD and the Agricultural Innovation System (AIS). Rationale for World Bank engagement in the FARA MDTF. The WB was instrumental in identifying the need for investment in regional and continental approaches to ARD in Africa, and in the design of programs and mobilization of resources to fund them. The Bank also provided critical technical support for institutional strengthening of FARA and Sub-Regional Organizations (SRO). The activities supported by the FARA MDTF were fully consistent with the WB strategy and priorities set in the Africa Action Plan. 1 Other Pillars: I - Sustainable Land & Water Management; II – Infrastructure & Markets; III - Food Supply, Hunger & Response to Food Crises. Page 6 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Theory of Change (TOC) The project’s TOC2 was built on the premise that a continental mechanism is needed to advocate for and manage support for ARD in Africa. The project aimed to achieve its two project development objectives (PDO) outcomes (Institutions aligned with FAAP principles and Africa’s innovation capacity strengthened) through: (i) E nhancing FARA Secretariat’s capacity to act as a continental ARD coordinator and project manager; (ii) Advocacy for increased investment into ARD institutions and programs at the country and sub-regional level; (iii) Capacity-building and promotion of ‘Innovation to Impact’ – pathways (IAR4D3); (iv) Building of continent-wide knowledge exchange systems and enabling environment; and (v) Support to regional and national level agencies’ capacity. Critical assumptions underlying the project included the principle of subsidiarity to guide collaboration among the key partners of FARA, financial support from DPs, sharing of resources for coordinated activities among partners, sustained commitment of partners to the regional agenda and sufficient capacity of National Agriculture Research Systems (NARS) and SROs. Figure 1. Reconstructed Theory of Change from the Original Project Documents (September 2009 and December 2013) 2 The TOC covers two main project periods: pre-restructuring (2009-2013) and post-restructuring (2014-2018). 3 IAR4D = Integrated Agricultural Research for Development - approach engages all actors, organizations and institutions that are involved in the agricultural sector to interact and jointly foster the development of the sector. Page 7 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Project Development Objectives (PDOs) The original PDO as stated in the Legal Agreement was to ‘align African agricultural institutions at the national, regional and continental levels with CAADP Pillar IV FAAP Principles for effective research, extension, and training and education’. Key Expected Outcomes and Outcome Indicators The key expected outcome was African agricultural institutions at the national, regional and continental levels are aligned with CAADP Pillar IV FAAP Principles. The project’s success was to be measured against four PDO outcome indicators: (1) Number of national and regional institutions supported by FARA that reflect FAAP principles in their ARD programs; (2) Percentage of NARS and SROs that indicate satisfaction with FARA’s contribution to the implementation of CAADP Pillar IV process; (3) Strategy for strengthening national and regional institutional and program capacities for ARD developed & implemented; and (4) Percentage of results of activities achieved. Components The project had the following four components: Component 1. Networking Support Functions (NSFs) (original US$10.43 million). The aim of this component was to mobilize and support FARA’s constituents and partners to undertake activities that generate continental spill overs and pro bono innovations. It had five sub-components. NSF 1. Advocacy and Policy (US$3.37 million) focused on supporting members establish institutional arrangements for ARD. The sub-component financed the CAADP Pillar IV Expert Reference Groups to engage NARS in CAADP roundtables, coordination among the SROs (ASARECA, CORAF and later the new CCARDESA and NASRO), NEPAD, CGIAR and other international ARD institutions, technical assistance (TA) to implement innovation platforms (IP) and strategic alliances4. NSF 2. Knowledge Sharing and Dissemination (US$1.99 million) supported researchers and end users through access to information, learning opportunities, and new technologies in agriculture and through dissemination of guidelines, communication strategy and advisory services. NSF 3. Regional Policies and Markets (US$1.61 million) promoted agricultural policy analyses & market research and support the application of research outputs in policy formulation. The aim was to provide Africa’s policy makers with policy options - empowering the continent’s representatives involved in international trade -environmental treaty negotiations. NSF 4. Capacity Strengthening (US$1.40 million). This sub-component assisted to help Africa attain sufficient human and institutional capacity for agricultural innovation via addressing research capacity needs and professional development of scientists. The activities included pilots to strengthen institutional and human capacity and TA to integrate capacity strengthening in CAADP processes and productivity programs. TA was also provided to facilitate North- South (N-S) and South-South (S-S) collaboration in higher education, teaching-learning methods and soft skills required for partnerships. NSF 5. Partnerships and Strategic Alliances (US$2.07 million). This sub-component facilitated establishment of mutually 4 An innovation Platform is a coalition of relevant ARD stakeholders (researchers, advisory service providers, agribusiness, policy makers, and farmer representatives) who associate to address specific ARD problems and opportunities. It is an institutional innovation that is envisaged to become the norm for how ARD actors organize themselves. Strategic alliances are alliances of likeminded actors at the continental level. Page 8 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) beneficial partnerships, capacity and platforms that draw on the resources and expertise of FARA stakeholders. The activities included information on sustaining effective ARD partnerships, identification and dissemination of proven partnership approaches for smallholder engagement and support to strategic ARD alliances between Africa and other regions. Component 2. Corporate Governance Reform Stream and Program Management (US$21.6 million). The objective of this component was to facilitate the reform, change management and capacity of the FARA Secretariat. This included the strengthening of internal management, governance and accountability of the Secretariat and to improve its support to the partners. Component 3. Sub-grants for Research Sub-projects (US$13.9 million). This component provided subgrants to beneficiaries for specific development projects involving research and advocacy in support of NSFs. Approximately 30.3 percent of funds were to be implemented by SROs, NARS, agricultural ministries, agencies responsible for leading CAADP roundtables, sub regional agricultural policy organizations, and secretariats of sub-regional and regional higher education fora. Component 4. Management and Supervision of the Trust Fund (US$1.7 million). This component covered the cost of the World Bank to manage, administer and supervise the MDTF. It was agreed with the contributing donors that the cost of managing and supervising the Project will be financed through the proceeds of the MTDF. Page 9 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets The revised PDO was to: ‘Strengthen Africa’s capacity for agricultural innovation with purposefully determined outcomes, creating capacity for innovation and enabling environment for implementation’. Accordingly, the revised outcome was: Africa’s capacity for agricultural innovation is strengthened. Table 1 provides a summary of PDO, outcome and PDO indicator changes. Following the changes in the PDO and outcomes, PDO indicators were also revised. Additional detail is provided in Annex 1. Table 1. Changes in PDOs, PDO Outcome and PDO outcome indicators throughout the FARA MDTF Original PDO Revised PDO (Dec 13, 2013) Key Change in PDO To align African agricultural institutions at the national, To strengthen Africa’s capacity for A shift from CAADP-FAAP alignment of regional and continental levels with CAADP Pillar IV FAAP agricultural innovation with purposefully programs toward overall improvement Principles for effective research, extension, and training and determined outcomes, creating capacity in innovation capacity education for innovation and enabling environments for implementation Original PDO Outcome Revised PDO Outcome Key Change in PDO Outcome African agricultural institutions at the national and regional Africa’s capacity for agricultural innovation A shift from country-region levels have aligned their ARD programs with CAADP Pillar IV is strengthened alignment of ARD agendas and FAAP Principles programs with FAAP toward a change in actual innovation capacity in Africa (diverse topics) Revised Indicators (June 28, Key Change in Indicators Revised Indicators (Dec 13, 2013) Key Change in Indicators 2012) Number of FAAP compliant A shift from engagement Percentage increase in number of A shift away from country-regional CAADP country compacts of countries in CAADP- individuals, groups, organizations directly level to focus on individuals and addressing agricultural FAAP to a change in affected or reached by FARA interventions organizations. research, advisory services, behavior (adoption of (disaggregated by gender) and higher education FAAP) Number of FAAP compliant A shift from satisfaction of Percentage increase in core competencies, A shift away from country-regional regional CAADP compacts NARS and SROs to a capabilities and capacities for innovation level to a change in innovation addressing agricultural change in behavior among targeted (individual, capacity among targeted individuals research, advisory services, (adoption of FAAP) organizational/inter-organizational and/or and organizations. and higher education signed. institutional) ARD actors % of AWP programs/ New Degree of stakeholder satisfaction with New - measures indirectly quality of activities implemented on FARA performance and quality of products services and accountability to FARA’s target, on budget and time and services constituency. Level of annual contributions by African New Governments and institutions to FARA funding (as % of FARA revenue) Page 10 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Revised Components The Network Support Functions (Component 1) were consolidated into three Strategic Priorities (see Table 2 below). Component 3 – subgrants for subprojects was modified in scope to focus narrowly on financing sub-projects on capacity building, networking, advocacy and training for sub regional organizations. Table 2. Summary of Revised Components Original Component 1: Networking Support Functions Revised Component 1: Strategic Priorities NSF 1 - Advocacy and Resource Mobilization Enabling environment for implementation of African agricultural innovation NSF 2 - Access to Knowledge and Technologies  Maintain knowledge hub to support strategic policy formulation NSF 3 - Regional Policies and Markets  Facilitate policy analysis and advocacy  Strengthen and connect African policy and economic research institutions NSF 4 - Capacity Strengthening Integrating Capacities for Change  Develop mechanisms for interactions and collaboration among different stakeholders  Improve the responsiveness and relevance of African institutions  Strengthen human, organizationaland institutionalcapacities NSF 5 - Partnerships and Alliances Visioning Africa’s agricultural transformation  Promote analysis to guide investment  Promote inter- and intra- continental collaboration of visioning platforms  Determine how the sector should be transformed Original Component 3: Subgrants for subprojects Revised Component 3: Sub-projects on capacity building, networking, advocacy and training for SROs Other Changes Other changes to the project included: reallocation of funds, extension of closing date and additional financing, change to environmental category and scope of activities. The changes were as follows5: 1st Restructuring: - Reallocation of funds between expenditure categories to avoid imbalance. For details, see Table in Annex 8. 2nd Restructuring: - The project closing date was extended from December 31, 2013 to December 31, 2018. Additional financing in the amount of US$45 million was approved. The original MDTF was prepared with expected donor contributions of US$45.9 million. However, only US$33.5 million were committed and a grant in the amount of US$31 million was disbursed by the original closing date of December 31, 2013. - Change to environmental category Environmental Category rating was changed from B to C - no safeguards policies were triggered under the MDTF. See section IV.B. for details. - Scope of activities: sub grants component 3 had a more limited scope (from original US$13.9 to US$1.89 million). Rationale for Changes and their implication on the original Theory of Change The 1st restructuring. The main rationale for the changes (Result Framework, PDO indicators, grant reallocation) was to: (i) ensure FARA can continue its work program activities uninterrupted; and (ii) ensure adequate reporting against a more clearly defined performance measurement framework, data collection and ownership of the indicators by FARA NSF Directors. These changes had no impact on the TOC. Regional Vice President (RVP) approval was received on June 28, 2012. 5 FARA MDTF Restructuring Paper - June 28, 2012 and FARA MDTF Project Appraisal Document (Additional Financing) – December 13, 2013. Page 11 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) The 2nd restructuring. In December 2013, the project was restructured to respond to changing context in African science and agriculture setting (transformation agenda, a widened mandate covering private sector engagement, youth & gender), increased capacity of FARA’s main partners, changes in resource availability and lessons learned during the implementation of MDTF6 (For details, see Annex 8). RVP approval was received on December 13, 2013. The PDO outcome indicators and the entire RF were adjusted to reflect the changes in FARA’s new Strategic Priorities and to better measure changes. Changes to Component 1 (Table 2) were envisaged to respond to FARA’s revised strategic plan. The Plan focused on operationalizing Africa’s vision for agricultural transformation, human and institutional capacity building, and provision of evidence for policy making. Component 3 was significantly reduced. This was in response to DfID’s withdrawal from supporting the MTDF and the fact that AFAAS and CCARDESA secured their own standalone MDTFs (from the European Commission (EC)). The TOC remained the same. However, FARA’s engagement strategy was adjusted with greater emphasis on strategic leadership and foresight, collective action on priority areas for regional solutions, and strengthening the capacity of national programs. These changes in engagement strategy, along with a new PDO and RF, strengthened the causal chain to some extent between the activities, outputs, and outcomes. 6The lessons included: a need for stronger engagement with policy actors and the private sector; the opportunity and risk presented by rising youth unemployment; and the need for Africa to develop the capacity to envision its future agriculture and food systems (the domain of foresight). Page 12 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) II. OUTCOME A split evaluation was used in the assessment of outcome achievement before and after restructuring in 2013, the time when the PDO and expected outcomes were significantly reoriented. A. RELEVANCE OF PDOs The relevance of the PDO is High. The PDO is still in alignment with the AU-NEPAD’s CAADP vision for how to reach the goal of 6 percent per annum growth for the sector. Since its approval in 2014, the Science Agenda7 for Agriculture in Africa (S3A) has been the overarching strategic framework to guide the ARD investments and broad areas of science that have to be developed by the African countries, their stakeholders and partnerships. CAADP provides the organizing principles in which the S3A can be operationalized. The S3A, therefore is the broader framework for the implementation of the FAAP. S3A also provides African decision-makers with the rationale for increased investments in science and technology. The PDO is still fully aligned with the World Bank agriculture strategy - given its priority areas include investments into agricultural productivity, climate change, human capital and knowledge. The WB was responsive to changes in the wider African science context (Annex 8), financing and the lessons learned during the MDTF, and subsequently restructured the project, including the PDO, restuls framework and components along with financing and extension of the project end date. Assessment of Relevance of PDOs and Rating Considering the above, there were no shortcomings in the relevance regarding the current context or responsiveness of the WB. The PDO remained Highly Relevant throughout the trust fund (TF) implementation (2009-2018). 7 The strategic thrusts of the S3A in the short to medium term are: the implementation of CAADP; increase domestic public and private sector investment; creating the enabling environment for sustainable application of science for agriculture; and to double current level of Agricultural Total Factor Productivity (ATFP) by 2025 through application of science. Page 13 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome8 The PDO outcome was twofold: (i) Alignment of ARD programs of African agricultural institutions at the national and regional levels with CAADP Pillar IV FAAP Principles; and (ii) Africa’s capacity for agricultural innovation is strengthened. 1. 2. Pre-restructuring 3. The pre-structuring phase project outcome was ‘ Alignment of ARD programs of African agricultural institutions at the national and regional levels with CAADP Pillar IV FAAP Principles’ – Rating is Substantial. Under the project, alignment with FAAP principles means ‘Country and Box 1. Alignment with FAAP Principles – process Regional Economic Community (REC) compact signing’. Alignment with FAAP principles refers to ‘Country-REC compact The progress on alignment was measured by two signing’ which is a process covering advocacy for a planning process indicators. At the country level ‘N umber of FAAP (by FARA), key country level stakeholder engagement and consultation, and signing of the compact to show commitment to compliant CAADP country compacts addressing CAADP-FAAP process. agricultural research, advisory services and higher education. The project supported 30 country compact After signing, the second part of the CAADP process entails follow-up preparation processes (against the target of 37) stock-taking, identification of national aspirations, and development of through advocacy and stakeholder engagement (Box a country investment plan (CIP). The CIP part involved e.g. guidelines, training, identification of CAADP focal points and facilitation of lessons 1). At the regional level the project was envisaged to learned (by FARA) to integrate AIS into national program. SROs often ensure compliance of regional compacts and have 6 participated in the 2nd part of the CAADP process1 (see Annex 1). regional compacts signed. However, the progress on regional compacts was modest and only 2 RECs were supported. For the alignment, three pre-requisites had to happen as intermediate results: (i) The process of signing country and regional compacts was supported by advocacy, Expert Reference Groups (on Research, Extension, and Education) for the CAADP Roundtables, stakeholder meetings and TA to strengthen NARS and REC capacity. The targeted countries were committed to CAADP-FAAP demonstrated by the signing of a compact and their country programs and investment plans reflecting FAAP (including IAR4D). Many of the six RECs on the other hand were reluctant and did not have the capacity to align their ARD programs with FAAP. In hindsight, RECs were not the right type of institutions to take a lead in a technical agenda. Due to overall delays in start-up, implementation, and funding, all alignment centered activities were not fully completed9 - affecting achievement of the outcome. The achievement rates for country and REC compacts are 81 percent and 33 percent respectively (Annex 1). Subsequent to Compact signing, the project supported 33 countries (against a target of 24) to develop Country Implementation Plans (Box 1) which are funding proposals that integrate CAADP Pillar IV elements. (ii) Capacity for alignment and agricultural innovation. The project activities contributed to the PDO outcome by improving stakeholder’s awareness, understanding and knowledge of CAADP-FAAP and AIS. This created stronger capacity by participating institutions to implement partnerships and IARD approaches (aligned with FAAP). The project had a two-pronged approach to capacity-building: capacity-building initiatives and implementation of Innovation Platforms. Page 14 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) (ii.a) Capacity-building initiatives. FARA carried out needs assessments (in 2006 and 2012) and implemented 13 capacity strengthening (e.g. on organizational performance, Tropical Agriculture Platform) initiatives (target 12) resulting in improved organizational performance among 71 percent of the targeted Box 2. Bilaterally funded capacity-building projects - alignment with FAAP Principles The SCARDA (Strengthening Capacity for Agricultural Research & Development in Africa) organizations (target 80 percent). As a result of program is the capacity-building project best aligned with the FAAP principles. SCARDA improved capacity, 3 (target 4) NARS (2008-2010) aimed to improve the capacity and performance of participating 12 NARS in 10 countries in key areas of their AR4D functions. While most activities focused on degree demonstrated change in capacity and adoption programs, it also offered short courses on such topics as farmers’ participatory research, of new IAR4D approaches. Most of the capacity- AIS approaches and IPM. The project was not extended beyond the pilot phase due to building activities were bilaterally funded Time inability to mainstream the approach into regional programmes (apart from CORAF). It was also constrained by ‘high transaction costs' associated with the subsidiarity Bound Activities (TBAs) that FARA promoted arrangements – requiring a long partnership process and agreement with independent through the SROs or directly at the country level organizations with varying capacity, with implications for pace of implementation and (Box 2). Former studies10 and interviews underachievements in strengthening of NARS. conducted for the ICR verify that, FARA played Interventions of the SROs were aligned with the FAAP from 2007-2008 onwards. Also, the a leadership role in supporting select countries EU-funded ICART (Implementation and coordination of Agricultural Research and Training) project, carried out situation analyses of the NARS (as per FAAP) in all 15 SADC Member to align their ARD programs with FAAP. States. Thereafter, the SADC Multi-country Agricultural Productivity Program (SADC- However, SROs also played a key role in the MAPP) adopted the FAAP, eventually leading to the creation of CCARDESA and APPSA Roundtable processes, along with bilaterally projects. The existence APPSA, WAPP, and EAPP are evidence that the FAAP had been adopted and put into practice by the NARS. The follow-up phases, WAATP and EAATP are funded capacity-building projects as a response aligned with the FAAP principles. to FARA advocacy on FAAP. Attribution to FARA Source: SCARDA End-of -Pilot Phase Report. FARA 2011, and FARA Secretariat. MDTF alone is therefore challenging. (ii.b) Innovation platforms and other strategic partnerships. With respect to improving support for agricultural IPs in Africa, the project exceeded all targets. It established 66 functional agricultural IPs (target: 51), 27 institutions (NARS, SROs, Farmers Organizations and Private Sector) adopted the IP approach against a target of 20. Sustained country demand for the IP approach reflects a positive change in awareness and capacity on AIS at the country level, and thereby, interest and capacity to commit to FAAP principles. (iii) Access to knowledge and decision support. In order to enhance access to knowledge and technology for African stakeholders the project facilitated access to diverse knowledge products and platforms11. In doing these, the project exceeded its target in enabling stakeholders to use gender sensitive knowledge management tools(target 2500 people, achievement 2821). The project made modest progress in ‘strengthening decision- making options for policy’. While there is limited evidence on the actual application of the recommendations from policy analysis and dialogue, the project produced 19 policy briefs (target 33) and provided 81 (target 60) policy recommendations synthesized from policy dialogues. Evidence from previous studies indicate that NARS and SROs benefited from wide-spread access to knowledge, technology, briefs, participated in planning and priority setting processes, and that 70 percent use those products. Improved access and use of knowledge and technologies contributed to creating awareness, understanding of and ability to integrate CAADP into country programs. 8 The RF did not cover components 2 and 3 - those results are covered in other relevant sections (II.C, III and V). 9 Alignment centered interventions continued during the post-restructuring phase. Appreciation for the FARA supported (with or without SROs) CAADP-FAAP integration into country NAIPs has grown, reaching more than 40 countries and 3 SROs. 10 Joint Review (EPMR and MTR) of FARA (2012), Strategic Review of DEVCO support to African continental and sub-regional agricultural research organizations (2016), Institutional Analysis/Science for Agriculture in Africa – Analysis of Consortium Partner Institutions (2017). 11 Knowledge products: 6 gender sensitive continental ARD knowledge platforms (target 6), 11 innovations via technology dissemination platforms (target 12), 29 national information and learning systems (target 25), 48 country knowledge platforms into continental platforms (target 25). Page 15 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Based on the evidence presented above, the project outcome efficacy is rated Substantial during the pre-restructuring phase. Post-restructuring The post-restructuring project outcome was ‘Africa’s capacity for agricultural innovation is strengthened’ – Rating is Modest. The outcome included two elements: the outreach/coverage of FARA-led interventions and improved skills and knowledge for innovation among stakeholders12. Outreach among farmers was 89 percent (substantial), among Box 4. Post-restructuring: two main PDO outcome indicators professionals and practitioners 82 percent (substantial) and countries 73 percent (modest). PDO1 - Percentage increase in number of individuals, groups, Capacity for innovation was to be measured by organizations directly affected or reached by FARA interventions percentage increase in number of individuals, Farmers: Target 1,786,555, Achievement 1,589,175 Professionals & Practitioners: Target 244,876, Achievement 201,179; groups, organizations directly affected or reached by Countries: Target 56, Achievement 41 FARA interventions, i.e. outreach. Second aspect of improved capacity is related to the improved skills PDO2 - Percentage increase in core competencies, capabilities and capacities for innovation among targeted (individual, and knowledge for innovation, which implies the organizational/inter-organizational and/or institutional) ARD actors behavioral change. The outcome was to be measured % Change in organizational capacity: Target > 90, achievement 69; by percentage increase in core competencies, Change in human capital indices: Target 16,750, Achievement 14,500 capabilities and capacities for innovation among targeted (individual, organizational/inter-organizational and/or institutional) ARD actors. Organizational capacity was improved 77 percent (modest) and change in human capital 87 percent (substantial). For the capacity for innovation to strengthen, three pre-requisites had to happen as intermediate results: (i) Capacity strengthening. With respect to improving innovation capacity, the project pursued for capacity assessments among 254 organizations (target of 300), supported capacity building initiatives13 among 136 ARD organizations (target 150) and established four Community of Practices (CoPs) on Foresight, Mechanization, Innovation systems and Biomassweb (target of 3 CoPs). Traction on CoPs has been reported good. Farmers, professionals and practitioners and countries were either affected or reached by these and other FARA interventions (see Box 4). The underlying assumption is that by participating in (consultations, workshops, IPs, incubators, trainings) or having access to outputs (advocacy, knowledge, platforms), the participants capacity for innovation is strengthened. Former studies- interviews verify improvements in participation-inclusiveness14. Organizational capacity and human capital were also improved by 77 percent and 87 percent, respectively. FARA implemented a range of projects that addressed different aspects of organizational and human capacity in ARD15. However, most trainings were of management skills (not core innovation capacity). Stakeholders’ views (as per ICR interviews) on capacity-building were positive and pointed to 12 Agricultural Innovation capacity - covers e.g., Assessment, planning and policy skills; Capacity for coordination & collective action; Education and Training for AIS; Soft and hard skills for research, extension, and private sector engagement; and Partnerships and business development. 13 Enhanced knowledge, skills and attitudes of individuals delivered through training workshops; changes in organizational design and culture, accountability, responsiveness, transparency and efficiency. 14 NARIs actively – together with SROs - participated in priority setting and strategic planning meetings, projects, IPs and partnerships, and capacity-building activities. NARIs and SROs also had representation in FARA Board, General Assembly and scientific events. 15 SSA-CP supported research capacity in collaboration with CG Centers; SCARDA addressed the research capacity of individuals and institutions (78 people trained at MSc level); PSTAD with components of DONATA and RAILS addressed capacity for Research-Extension linkages of IPs. FARA has also contributed to the SCARDA approach, applied in the TAP (now TAP Common Framework for CD for AIS). Page 16 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) demand for continued efforts (e.g., NARS capacity-building, revision of curricula). An independent institutional analysis conducted in 2017 on FARA, however, provided contradictory views – where 55 percent of FARA constituency stated that FARA’s capacity-building interventions need improvement. During 2015-2017 in particular, low capacity and changes within SROs themselves affected implementation (e.g., the AHC-STAFF initiative) significantly. Capacity- building partnerships, however continued, some being scaled-up by SROs16. (ii) Innovation Platforms and Partnerships for collective action on AIS agenda . Partnerships create important opportunities for coordination and collective action which are at the heart of strengthened AIS capacity. The project supported hundreds of IPs but fell short of achieving the target (859 against target of 1000)17 in diverse ARD projects, namely the Sub-Saharan Africa Challenge Program (SSA CP). The demand for IPs by countries and SROs is steadily growing, indicating improved awareness, experience/capacity and (hopefully) sustainable scaling-out of AIR4D approaches. The project has also reached or trained thousands of farmers and professionals on these approaches (IP targets achieved at 71 to 103 percent level). Previous studies and the ICR interviews indicate that collaboration has generated better skills and appreciation for benefits arising from collaboration (among NARIs), albeit slowly. SROs are also better at collaborating with their constituency. There is still significant room for improvement in overall coordination of AIS activities. While FARA was instrumental in advocacy for IAR4D, S3A development and support for IPs, it often worked in isolation and sometimes implemented activities directly with NARS bypassing the SROs. SROs, on the other hand, focused working with their constituency and did not engage in cross- region collaboration. (iii) Knowledge and enabling environment for AIS. In order to enhance ARD stakeholders’ access to and use of relevant knowledge for awareness and decision-making for investments into AIS, the project produced 190 knowledge products (against target of 212) and maintained knowledge sharing (reached 119 percent of target) and other exchange platforms (60 percent of target). About 41 countries (target 15) used foresighting studies in development of their ARD agendas. Despite seemingly good results, feedback (from the independent Institutional Analysis 2017) pointed to limited results with the knowledge management activities. The project adopted a strategy to increasingly rely on ICT for knowledge sharing. FARA constituency was satisfied with Regional Agricultural Information and Learning Systems (RAILS) and improvements in ICT connectivity (Connectivity improved in 34 ARD organizations, target 41). The revised PDO (Strengthened agricultural innovation capacity) was overly ambitious and hard to measure – given innovation capacity (in Africa) term covers a wide spectrum of interventions, mostly not supported by the project. Although the project had some positive achievements, the post-restructuring outcome achievements is rated Modest. Justification of Overall Efficacy Rating Rating for Overall Efficacy is Substantial. The operation achieved most of its intended outcomes of Alignment with FAAP Principles and Strengthened capacity for agricultural innovation. 16 The formulation of the regional WAAPP, EAPP, and APPSA serve as examples of capacity-building projects scaled up by the SROs and countries. The countries involved were initially engaged by FARA-led projects such as PSTAD (DONATA & RAILS) and SCARDA. The SADC secretariat extended the approach to EU-funded projects such as ICART and the French-funded collaborative research support project FIRCOP. The institutions involved continued interventions within the context of CCARDESA, CORAF and ASARECA, leading to the evolution of the IP work into the IAR4D. 17 Target level raised many times during the implementation. Page 17 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) C. EFFICIENCY Assessment of Efficiency and Rating The assessment of the project efficiency focuses on: (1) evaluating overhead costs and relevant comparisons, and (2) examining resource use efficiency, examples and feedback. The project’s Efficiency rating is Modest – given the fact that efficiency in resource use is below expectations. Overhead costs. The share of FARA ’s Secretariat costs to total project cost were high (44 percent of total funding). Only US$14.0 million (31 percent) was allocated for supporting technical programs. The secretariat was a large organization (60+ staff) with FAO level salaries and benefits. Despite significant streamlining of the secretariat (e.g., FARA’s staff had been reduced from 60 individuals to 38 by January 2016), technical programs, and cost-sharing with SROs (personnel) from 2015 onwards, the proportion of overhead costs remained high at 69 percent (Data in Annex 4). Comparison to relevant organizations. Recommendations for operating costs for not-profit organizations range from 60 -80 percent, leaving 20-40 percent to overhead including salaries. In private sector (2013) overhead averages 22 percent - but may be 50 percent if the company provides professional, scientific and/or technical services (including consultants)18. By any account, FARA’s overhead costs significantly exceeded generally accepted recommendations. Comparable regional ARD-innovation coordinating forums (such as Foragro, SCAR Europe, and APAARI – see Annex 4) have lighter operational structures. The overhead costs were expected to be higher than the norm in the early stages of the MDTF. In the absence of a functioning network, FARA contributed to building of a regional network and carried out many activities initially itself. FARA also received a total of US$117 million from bilateral donors (Annex 4), reducing the overall overhead costs to 24 percent. Charging for all TBA associated overhead costs would have been a prudent approach19. Resource Use Efficiency. FARA’s resource use was not efficient during the MDTF phase. Cost efficiency was not strongly applied. In addition, unfamiliarity with WB procedures and lack of results-based performance culture within FARA contributed to delays and inefficiencies. In 2014-2015 the Bank had significant concerns over high operational costs and lack of adequate value for money for some expenditure. The views of the constituency20 also varied regarding ‘value for money’. Most stakeholders stated that the FARA MDTF was Fair to Good value for money, given the MDTF allowed FARA to focus on its mandate (until TBAs took over). Given the fiduciary challenges and value for money concerns, FARA gradually instituted mechanisms (e.g., streamlining of secretariat, cost-sharing, improved cost efficiency) for increasing efficiency of resource use. Efficiency (or lack thereof) is illustrated by three examples (Annex 4). Efficiency in achieving the PDO outcomes is Modest given many of the project interventions did not contribute to achieving the PDO outcome and the overall lack of efficiency in resource use. D. JUSTIFICATION OF OVERALL OUTCOME RATING The overall outcome rating is Moderately Satisfactory. Although at a borderline rating, taking into account the 18 The Better Business Bureau's standards; Netplaces.com; Second Wind Consultants. 19 FARA was often not able to recover the overhead costs from the TBAs. Some donors did not allow this – rationale being the MDTF core funds. 20 Based on a stakeholder survey – subjective statements. Page 18 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) relevance of the objectives and efficacy and the evidence that the project made substantial progress in achieving its results during the first phase of the project, which accounts for 66.4% of the total disbursements, a Moderately Satisfactory rating is considered to be a reasonable assessment of the Project’s overall outcome. The table below summarizes split assessment of outcome achievement. The table below summarizes split assessment of outcome achievement. Pre-restructuring Post-restructuring Relevance of Objectives High High Efficacy Substantial Modest Efficiency Modest Modest Overall rating Moderately Satisfactory Moderately Unsatisfactory % disbursed 66.4 33.6 Overall rating Moderately Satisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender Gender mainstreaming activities began late (e.g., Gender Specialist hired in 2013) but the project was able to positively influence SROs and many NARS (trainings, IPs and Incubators with a gender focus, knowledge platforms, gender disaggregated monitoring and evaluation (M&E)) on their gender practices (as per Stakeholder Interviews). In the Humidtropics project, at least a third of the participants in training and other actions were women. FARA also produced a guidebook entitled ‘Gender Mainstreaming in Innovation Platforms’ that is being used to develop a training toolkit for gender mainstreaming within the IPs. The engagement of the youth was also prioritized. The sustainability of the IPs were improved due to their inclusivity. Mobilizing Private Sector Financing The IP approach has been able to leverage partnershipsthat integrate private sector investments into commodity value chain development while generate innovations with socio-economic benefits. The list of various IPs are on the Innovation Platform Agribusiness Portal. http://ipabp.org/ Poverty Reduction and Shared Prosperity While poverty reduction was not an objective, the FARA MDTF has had an impact on adoption of technologies and incomes among participating households. Empirical assessment from various FARA initiatives indicated that: adoption of technologies increased by 50 percent (DONATA, FARA), incomes may double (IAR4D, FARA), IP approach has reached 800 households/farmers as direct participants (IAR4D, FARA) and each UniBRAIN Incubator can create 300 direct jobs per year (UniBRAIN, FARA). In addition, empirical evidence from 25,000 household survey (SSA CP) indicated that the average income of individuals participating is US$1,362.72 (an average of US$3.73/day) compared to the baseline income of US$588.44 per household (an avergae of US$1.61/day). This survey pointed to the following achievements: Income of women was increased by 326 percent, food security was improved by 324 percent, Page 19 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) household income increased by 232 percent, income inequalities decreased by 6 percent (Gini Coefficient) and productive assets value increased21. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION Africa-wide stakeholder engagement, commitment and vision for agricultural productivity guided the MDTF. Significant background work, including CAADP process, the development of FAAP and S3A, and harmonization efforts guided the two design phases (See Context and Relevance), creating significant momentum for investments in agricultural productivity. Parallel TFs supported SROsand AFAAS and three regional programs WAAPP, EAPP and APPSA were established to strengthen sub-regional capacity and out-reach to address the risks with implementation capacity. FARA was a relevant organization to lead the MDTF but its size was overestimated for the purpose. The project was to be led by FARA which was well placed to take the lead (See Context and Annex 7). The historical background however contributed to challenges with long-term sustainability and accountability to its constituency (see Efficiency, Lessons). The Secretariat centered governance and reform activities were detailed, receiving a substantial focus at entry. A high portion of the networking support functions (NSF) activities were continental in nature/were directly implemented by the Secretariat for other reasons, contributing to high overhead. The Project Document (PD) referred to salary review and adjustments to be made during the MDTF – such adjustments were delayed until the 2nd restructuring22. The PDO and the M&E design were not fully developed and coherent with negative spill-over effects on implementation. The many deficiencies point to entry level challenges in identifying priority results and how to reach them, incomplete understanding of results-based M&E, and potential confusion over the roles of FARA and those of the sub-regional and national stakeholders. The TFs were not required to develop a PAD - the lack of a fully developed PAD likely contributed to undefined objectives and incomplete components. Incomplete component design, especially of component 3 (recipient executed sub-grants) hampered the project take off. The WB guidelines and quality assurance process with respect to recipient-executed grants was limited. While the sub-grant approval process was covered in detail, the sub-grant activities received limited attention (demand-based nature). The grant agreement stipulated the withdrawal conditions, i.e. no sub-grants could be approved without prior Bank approval. The component activities were delayed for two years (see III.B). Adequacy of risk and mitigation measures. The PD rightly stated that many variables were outside the control of FARA whereas the activities under the Strategic Plan were to be within control of FARA. The following risks were not properly identified at appraisal, constraining the implementation and achievement of the outcomes. 21 https://faraafrica.org/wp-content/uploads/2014/07/Maximising_impact_from_agr_res_Ver07.pdf https://faraafrica.org/volume-2-no-13-2018-impact-of-the-integrated-agricultural-research-for-development-guidelines-on-smallholders- livelihoods-in-ssa-cp-innovation-platforms-revised-edition/ 22 InMarch 2008, FARA commissioned a comparative study of the Employment Environment, Staff remuneration and Benefit packages as part of MDTF requirements. The recommendations were: FARA’s salary and benefits package s are comparable with those of similar organizations in the African ARD employment market. It was recommended that FARA should maintain the policy of alignment of its salaries and benefits package with those of the UN system. In 2014, in order to fit within the provided staff budget and implement its programs, FARA undertook a job evaluation and adopted its own salary scale – with subsequent downgrading. Since the onset, FARA has not increased salaries. Page 20 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) a. Donor commitment to financing estimated as Negligible risk at entry (raised to substantial post- restructuring). The rationale for low risk was that DP commitment was expected to remain high however, out of four DPs, the EC remained as the only original DP supporting the TF throughout the project. In addition, commitments were not aligned with initial pledges for financing. b. No institutional sustainability risk considered at entry. Financing for FARA was expected to be available for years to come, and institutional sustainability of FARA and development of an exit strategy were not considered at entry. c. Capacity of implementing agencies with a modest risk, rather than high. Implementation relied a lot on SROs and NARIs – some of which did not yet exist or had limited capacity and/or resources. Capacity limitations exposed FARA to delays with implementation and reporting. The Project Document 2008 & Project Appraisal Document 2018 both stated that FARA secretariat may have resort to implementing many activities itself. This took place, resulting in challenges with adhering to subsidiarity as well as increased fiduciary risk. d. No risk with M&E capacity at entry M&E capacity within the Secretariat and the implementing partners received insufficient attention at the beginning. Complex multi-stakeholder and multi-timeline projects/programs on research-extension-innovation are in general known for having M&E challenges. Page 21 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) B. KEY FACTORS DURING IMPLEMENTATION Several factors, including the challenging financial situation, affected progress with the MDTF. The ratings for progress toward DO and overall IP were lowered to MS-MU until the end of 2018. Due to fiduciary and procurement constraints, implementation progress was rated as Moderately Unsatisfactory in 2016-2017. External Factors The initial delay (from 2008 to 2009) and more significantly the gap (of roughly US$12.4 million) in committed funding contributed to implementation delays. FARA made reasonable (satisfactory) progress in its overall implementation – yet most technical program activities were delayed by 1.5-2 years (see section II.A). The additional financing (from European Commission (EC)) and the extension of the closing date to 2013 alleviated the immediate constraints. Changing transformation agenda and S3A (led by FARA) affected FARA’s mandate and the MDTF. The development of the S3A, and its adoption by the AU Summit in 2014 as the guiding STI agenda for CAADP, triggered the restructuring of the MDTF in December 2013. FARA repositioned itself for S3A implementation which focused on mainstreaming of S3A into NAIPs, with subsequent efforts to spearhead implementation of S3A (advocacy, communication, social marketing and guidelines). The focus was now on the strategic activities which were better aligned with the continental mandate of FARA. Given only EC of the original DPs made a firm commitment of €14 million to support post-restructuring phase, high, medium and low budget scenarios23 were developed. FARA opted to implement the low case, against which the work plan 2014 was approved. Changes by the African Heads of State and limited resources prompted FARA to make informal changes to the entire project in 2016 (see Annex 8). In May 2014, the AU Heads of State issued a Malabo Declaration that shifted the focus of the CAADP Pillar IV to country level development and gave greater focus on the S3A as the overarching framework for the MTDF strategy. Due to the financial shortfall, FARA adjusted its focus, revised the entire RF and reorganized and renamed the components without the consent of the WB. Subsequently, the project targets were tracked against 2013 and in 2016 RFs, negatively affecting M&E and tracking of the original 2013 targets. Factors subject to FARA and World Bank control Challenges with M&E continued to constrain the project. Significant short comings with M&E at entry (see III.A and IV.A) had negative spillovers on M&E and overall reporting by FARA and the implementing partners. In practice, all reporting until the end of 2013 was against revised (in 2010) PDO and IR indicators, whereas officially the indicators were adopted after the formal restructuring in June 2012, creating confusion on M&E and reporting. The changes in key personnel and the overall leadership and performance culture within FARA affected the performance of the MDTF. FARA experienced many personnel changes. The new Executive director (from July 2013) had a different management style requiring adjustments from staff and partners. At times the Secretariat had significant difficulties in recruiting quality replacement e.g., for procurement, financial management, and M&E, and 23 The high case scenario was $121 million, of which $63 million was anticipated from the MDTF and $58million from bilateral sources. The other alternative scenarios envisaged expenditure ceilings of about 80% or $98 million and $ 55 from MDTF (medium) and 60% or $70 million and $46 million (low) of the high case scenario. Page 22 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) unclear reporting triggered procurement and fiduciary management reviews (see IV.B). Apart from personnel and management changes, FARA remained large and maintained a flexible, open-ended (not time or results-bound) staff performance culture, easily giving arise to delays. Implementation delays of component 1 and 3 plagued the first 2-2.5 years, while secretariat functions kept on drawing on the core MDTF funds. Components 1 and 3 were significantly underperforming and under-disbursing for the first 1.5 years24 which was consumed mostly by putting together project management basics. FARA’s limited capacity in procurement and general unfamiliarity with the WB procedures contributed to those delays. Component 3 continued to lag, mostly because of lack of sub-grant Operational Manual (OM) (revised and approved by FARA Board in 2011) and not having a sub-grant facility in place. The first sub-grant was issued to establish AFAAS in July 2011 and the second to CCARDESA in 201325. Insufficient prioritization and an expanding work program shifted attention away from FARA’s mandate. FARA’s technical program was not always aligned with ARD-innovation. The work plan was largely developed as per perceived needs of the sector and FARA continued to respond to new stakeholder requests for support throughout the MDTF. Implementation of new activities at times crowded out the agenda, with subsequent implications for budget and delivery. Many of the FARA led activities were also on-the-ground rather than at the continental level. Increasing financial challenges made it attractive (for FARA and SROs alike) to behave opportunistically rather than strategically and accept financing when available. Some DPs may be accountable for lack of strategic approach - many considered FARA a useful and trusted partner to manage and channel funding for their bilateral project beneficiaries. Thereby, FARA ’s agenda gradually shifted away from CAADP’s original focus (as expressed by many stakeholders). FARA’s resource mobilization was put in place late – and the overall strategy was not successful in an increasingly competitive financing environment. In the early stages, FARA benefited from the reputation of its Executive Director (ED) and was able to attract funding from three donors. By 2013, the financing conditions were significantly different, with one key donor EC (Annex 8). The Secretariat was to minimize the heavy reliance on DPs and instead mobilize resources from African governments and the private sector. After some delay and steep learning curve, the new ED engaged in resource mobilization efforts26. FARA’s resource use was however not efficient (e.g., 2014-2015) and its overhead costs were above the norm (see Efficiency). Over the years, FARA also faced increasing competition for resources both regionally by the SROs and continentally27. Many DPs questioned value for money and were not willing to support FARA in its then current size and expenditures. Both the secretariat and technical programs went through upheaval to survive in the new fiduciary conditions. FARA Board took the responsibility for economizing FARA’s size and activities. Post-restructuring phase featured a persistent need to streamline the secretariat through reorganization, cost-sharing, redefinition of role, and 24 Component 1 with a planned budget of US$1.5 and US$1.1 million in 2010 and 2011 respectively - no disbursement on component 3. 25 Eventually,very limited funding was allocated to this component (US$1.3 million by 2013, of a total of 1.89 million) given DfID funding did not materialize and subsequent resource mobilization was limited. DfID had concerns over FARA’s financial management in the early stages. Also, DfID’s overall development priorities changed with subsequent greater attention given to impact on the ground. After 2nd restructuring, most disbursement (US$400,000) on component 3 again targeted at AFAAS and CCARDESA. 26 AfDB, IFAD, BMGF, ACIAR, DFID, USAID, Netherlands, Italy, DANIDA, NORAD, CIDA, IRDC, BMZ, JICA - DANIDA, BMZ, ACIAR and IFAD preferred to make contributions through TBAs. Ghana, Nigeria, South Africa and Uganda have supported FARA in the form of one-time contributions. 27 AUC and African Agricultural Technology Foundation (AATF) funded by several donors, Alliance for a Green Revolution in Africa (AGRA) funded by Bill and Melissa Gates Foundation (BMGF), the Food, Agriculture and Natural Resources Policy Analysis Network (FANRPAN) funded mostly by BMGF, and AUCA (African Union Development Agency)-NEPAD by JICA. Page 23 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) prioritization of activities. FARA had to rely more heavily on its own staff28, rather than external consultants to undertake implementation. In this regard, FARA was further constrained to deliver continental outputs and outcomes envisaged in its SPs. Subsidiarity was a key element of FAAP and the MDTF – but remained a contentious issue (on behalf of both SROs and FARA) throughout the implementation. The assumption at entry was that SROs will implement many of the regional and country level activities. Some mixed level activity was also expected given FARA supported the establishment of regional and continental organizations. Due to lack of capacity and (sometimes) resources, FARA engaged in many activities to fully coordinate activities in many locations whereas the SROs were more focused on individual projects led by several project officers. With time, the concept of the NARS within FAAP was better understood, and a more integrated approach to implementation was understood by the SROs and countries. The relationship with SROs was often tense – mainly due to disagreements with subsidiarity and accountability but also because of capacity. The majority of FARA initiatives focused on developing and testing new approaches. In situations where SROs had only few pilot countries to choose from, the SROs occasionally recommended FARA to engage with the countries. SROs and FARA were/are effectively competitors rather than collaborators. FARA led the consultative development of S3A (de facto follow-up to FAAP alignment) as well as the Consortium that aimed at leading the follow-up implementation of S3A. Due to lack of traction among key partners (mostly SROs), the Consortium approach collapsed in 201729. There is an obvious need for country-level coordination of the multiple IAR4D activities, yet, the coordination function requires clarification and agreement among stakeholders. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design M&E Design at entry was weak, hinting unfamiliarity with results-oriented M&E. The original PDO was vague and hard-to-measure. The RF was incomplete (missing Components 2 and 3), often unaligned between outcomes and outputs, and mixing PDO, Intermediate Results and output level indicators. Thus, they were often either too ambitious and poorly or not sufficiently ambitious. The FARA Strategy and its component 1 related results provided an additional set of results, which were not fully aligned with the relevant section in the RF. While most indicators were quantifiable, they were rarely qualitative. All indicators had baselines and targets that were mostly easy to come by, i.e. most were strictly related to FARA-led actions/information FARA had easy access to. All M&E arrangements for data collection, timing and responsibility were defined, however, well intended approach cannot compensate poorly defined indicators or baselines. See Annex 10 for details on M&E arrangements throughout the MDTF phases. 28 Theeffect of using own staff had also positive outcomes: FARA had an opportunity to gather and share information on various platforms with stakeholders. FARA gained experience in implementing e.g., governance and fiduciary management and was able to pass this on to strengthening the SROs and establishment of CCARDESA and NASRO. The pace of implementation in some cases may have slowed down whereas quality was retained via participatory working approaches and attracted additional funding. 29 SROs were highly involved in the development and consultations of the S3A. Given the changes within the SRO leadership and technical staff, the institutional memory and ownership of the coalition have been partly lost. Some SROs treat S3A as a FARA led TBA rather than the main CAADP instrument for implementation at the country level. Among countries, traction for S3A is better. See lessons learned. Page 24 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) M&E Implementation The M&E implementation was a concern during the MDTF phase. Many of the challenges arose from quality of the M&E at entry and challenges in recruiting M&E expert (2011-2013, 2017) 30. The data providers were not clear on how to collect the data, as they were not consulted on the formulation of the original indicators. Limited capacity of the implementing partners also contributed to delays. Significant improvements to the set-up were made following corrective action informally in 2010 and as a result of the CIDA-sponsored Results-based M&E workshop in February 2011. Reporting against indicators began 1.5 years into effectiveness and all reporting until the end of 2013 was against revised PDO and IR indicators (set in 2010, approved in 2012). Following the FARA Board’s acceptance of the refined RF, the Bank formally restructured the project in June 2012. Since post-restructuring, the M&E Unit was better equipped with staff and a standardized multi-channel/stakeholder data collection and monitoring system. Due to changing circumstances (strategic needs, financing gaps), the basic tenet of the M&E was again revised in 2016, however, the changes in the indicators were not formally approved, and all reporting was against the 2014 set-up. Towards the end, the FARA unit worked in a cost-sharing arrangement with SRO M&E units, and efforts were made towards developing coordinated, harmonized M&E system for AFAAS, NARIs and FARA. See also section on Bank performance at entry and supervision (IV.C). M&E Utilization Overall, M&E data was used in planning (preparing annual workplans and MTOPs), in reporting achievements and in development of resource mobilization strategy. Especially during the post-restructuring phase efforts were made toward use of the M&E data in decision-making. M&E products were considered useful to understand progress, or lack thereof, in the implementation of the program and to make the necessary adjustments. Similarly, the M&E reports were useful tools for the Secretariat to make budgetary and programmatic decisions and provide data to improve FARA decision-making. In the projects e.g. SSA CP and SCARDA M&E data was used to analyse the outcomes and impact of the interventions these projects set out to test. Justification of Overall Rating of Quality of M&E The overall rating of quality of M&E is Modest. There were significant shortcomings in the M&E design at entry (2009), the project revised RF twice informally (2010, 2016) and twice formally (2012, 2013). Such changes were implemented to improve the M&E tracking and RF. The many changes also resulted in significant confusion on M&E implementation and challenges in tracking milestones. Initial 30 Theproject was without a dedicated M&E officer for nearly two years (2011-2013) and again in 2017. Procurement and financial management capacity varied largely depending on the availability of professionals but also due to the Secretariat’s cost-saving efforts. For example, the Bank supervision team insisted hiring of a procurement officer, resulting soon in improved procurement rating. Page 25 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and social safeguards At entry, the safeguard-screening category was S2, given the Environmental Assessment, Pest Management, Forests and International Waterways policies were triggered. Many of the specific activities to be supported through FARA’s NSFs were not known at the time. Similarly, environmental risks and any required associated management plans associated with the recipient-executed sub-grants were to be identified in submitted proposals, evaluated as criteria for sub-project funding, and the implementation of their environmental management plans to be monitored by FARA. Only two sub-grants (set-up and early operations of AFAAS and CCARDESA) were funded during the MDTF. No safeguards were triggered. Subsequently, the grant was restructured in December 2013, and the Environmental Category rating was changed from B to C. FARA was not to undertake any activities that would require physical or civil works. The restructuring, which was processed outside of the Operations Portal, was approved by the RVP with all the necessary documentation filed in WBDocs. Given the environmental category rating change was never formally reflected in the system, the OPCS was still (in July 2017) in the process of resolving the issue. Financial Management (FM) In terms of fiduciary compliance, the project complied fully with the financial covenants of regular submission of acceptable quarterly interim financial reports. With regards to annual audit reports, FARA has also fully complied with all statutory submissions. The audit report for 2014 included a management letter which highlighted some minor internal control lapses. Generally, these were not very significant or material to affect the true and fair view of the accounts. During the May 2015 mission, the Bank raised concerns related to accountability for cost control and value for money and undertook an In-depth Financial Management and Value for Money audit covering 2012-2014. The findings of the Review did not observe significant weaknesses or lapses in the FM systems. FARA’s 2015 annual audited financial statements were, despite some internal control lapses, unqualified and received on schedule on June 30, 2016. The recommendations of the review raised the following key issues: (a) concerns expressed by the Bank on the need for more scrutiny and fiduciary oversight of the use of project funds generally and particularly regarding expenditure of the ED’s Office; (b) submission of a formal response from FARA to the Bank advising how it intends to address the recommendations of the report; and (c) refund of ineligible expenditures. On August 2016, FARA submitted its formal response to the recommendations of the fiduciary review and by end August, the Bank issued an invoice to FARA to refund the ineligible expenditure identified by the review. The full refund was processed by the Bank on October 5, 2016. Based on the above action taken by FARA to address the findings and recommendations, the overall FM performance of the project was upgraded to Satisfactory (S) and the FM risk was reduced to Moderate. C. BANK PERFORMANCE Bank performance is Moderately Satisfactory. Page 26 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Quality at Entry The WB was instrumental in identifying the need for investment in regional and continental approaches to ARD, in the design of programs, as well as in the mobilization of resources. The WB also provided support toward building the capacity of FARA and other SROs to be able to develop and implement their programs. The Bank has played a key convening role with a core group of DPs and African institutions at every level. The MDTF was of high strategic relevance, addressed technical and financial aspects, and paid attention to gender and environmental aspects. The MDTF at entry however suffered from a weak M&E design, an unusually high allocation of resources (44 percent) to the Secretariat, somewhat undefined role for FARA, an incomplete project design, and underestimation of significant risks. Quality at Entry was Moderately Unsatisfactory given there were significant shortcomings in quality at entry. Quality of Supervision Over the lifetime of the MDTF, the project received significant supervision and evaluation input. Twenty supervision missions, two Mid Term Reviews (MTRs) (Jan-Feb 2012, June 2016) and two restructurings, including financing arrangements, were completed. The MTRs were used as important opportunities to take corrective action. The restructurings affected the performance in a positive manner. In addition, an informal Advisory Committee consisting of four DPs and the WB met initially bi-annually and contributed to the quality of the MDTF, including reviews of science agenda, recruitment, M&E, and restructuring of the MDTF. While the group had no decision-making power on supervision, it was instrumental in discussing the conditions, development and financing. In the early stages, the project design, resource allocation, and M&E were under a constant experimental learning by doing mode. The Bank was actively engaged in adjusting and supporting the take-off. The ISRs paid attention to FARA ’s sustainability and role. While WB provided significant support on M&E in 2010-2012 (see section on M&E), attention to M&E entry and in the first 1.5 years was not sufficient. The Bank should have advised FARA not to make informal revisions of indicators and RF in 2010 but to carry out a formal restructuring. The ISR ratings were also fairly high compared to very limited progress was made with implementation for the first 1.5-2 years after effectiveness. The financial risk associated with the 2nd restructuring was high. Significant challenges with resource mobilization were expected, yet an additional US$45 million of grant financing was requested (and approved). The project’s ‘mitigation strategy’ was to develop three funding scenarios (low, medium, high), and adopt the low scenario. The risks unfolded, with consistent streamlining efforts throughout the post-restructuring phase. During the 2nd restructuring period, the Bank provided proactive and candid technical feedback and support and made field level supervision missions. A lot of the supervision and information and knowledge sharing effort targeted at FARA’s survival, i.e. streamlining of the secretariat and its activities, providing information on organizational options, and guiding resource mobilization efforts. The Bank also initiated a fiduciary and procurement assessments in 2016. While M&E received attention, the entire RF was informally changed in 2016, resulting in two parallel RFs and difficulties with tracking milestones and assessing achievements. Bank initiated a 3rd restructuring in 2017 to formally approve the previously made RF changes, however, given the project was coming to an end, this activity was dropped. Page 27 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) In 2016-2017 Bank initiated an institutional analysis to address transition arrangements for the MDTF and continental coordination of IAR4D actors in general31. The review proposed a transition arrangement to the Consortium’s operational framework. However, the joint proposal (by SROs, AFAAS and FARA) was turned down by FARA’s Board due to concerns (over FARA’s prominent role) raised by SROs. Quality of Supervision rating is Moderately Satisfactory. Despite significant efforts on technical support and transition arrangements, there were moderate shortcomings in supervision during the MDTF and post-restructuring phases. Justification of Overall Rating of Bank Performance Given the more significant challenges at entry and moderate shortcomings at quality of supervision, the overall rating of Bank performance is Moderately Satisfactory. D. RISK TO DEVELOPMENT OUTCOME The risk to development outcome, i.e. sustainability, can be viewed from many perspectives. A selected set of issues are viewed in Table 3 below. Annex 9 provides additional details on select project initiatives. Table 3. Risk to Development Outcome in the FARA MDTF Focus Sustainability Higher level - Some increase evident in government commitment to IAR4D – investments have increased 47% from 2000 Government to 2014. Some governments have also allocated resources for integrating S3A into country NAIPs. commitment to investment, productivity, The MDTF has been able to reach thousands of female and male farmers indirectly and directly primarily growth – via IPs and incubators. Adoption of new technologies, improved incomes and skills are evident among Risk – Modest to High. many actors (farmers and service providers). Farmer adoption - Risk – Modest. PDO - Alignment of More than 40+ countries and SROs have adopted FAAP principles in their country NAIPs. Some are already countries and regions taking steps towards integrating S3A and developing 2nd stage NAIPs. Interviews indicated appreciation for with FAAP-CAADP (and this process and commitment to integrating S3A either with their own resources or with FARA-SRO S3A) generated resources. The SRO commitment to S3A is still lacking while most countries, FARA and AUC show Risk - Modest. commitment. PDO - Capacity of African The basic AIS elements include coordination and collective action; new and old skills of the AIS agricultural innovation professionals to serve in different functions; research & extension systems with skills, resources and system inclusive IAR4D approaches; engagement of private sector; and an enabling environment. The MDTF focused on a select set of AIS activities. Risk - Modest. Culture of collaboration, coordination and trust among the stakeholders are essential- while FARA and SROs were able to collaborate well on selected initiatives e.g. IPs, their collaboration suffered from limited trust and competition. Collaboration centers currently on individual SROs working with their country constituency. FARA and NARI level collaboration is smoother and centers on CAADP-FAAP, IPs and capacity- partners to achieve the proposed 31 The institutional analysis was to assess the existing resources and capacities of the consortium objectives of the Consortium and provide recommendations on programs and the Consortium’s architecture. Page 28 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) building. While FARA has raised the profile of AIS stakeholders and supported change, the continental level coordination is still lacking. Strategic partnerships, more inclusive IPs, agribusiness incubators for technology dissemination, improved access to knowledge and technology, and diverse capacity-building initiatives have improved capacity in many locations. The sustainability of these changes will depend on their continued scaling out and widespread adoption. The MDTF has reached many organizations, professionals and farmers, potentially serving as champions for change. NARIs have embraced capacity-building activities, including the need to address current education curricula. A coordinated effort is required among governments, core AIS stakeholders (including private sector) and development partners to invest in priority areas of AIS, including enabling infrastructure and regulatory matters (such as seed and fertilizer sector) that currently constrain technology adoption, private sector engagement and value-added activities. Key Outputs-Initiatives IP approach has generated good results. The MDTF has also developed guidebooks for this process. The approach/es have been well adopted by NARIs, SROs, bilateral projects, and CGIAR CPs. The practice has Risk – Modest. high demand in countries, is treated favorably and likely to remain an active part of the AIS portfolio. Danida funded and FARA developed UniBRAIN activities were transferred to a self-sustaining entity AAIN that scales out this initiative. New organizations - AFAAS and CCARDESA are fully established and can function independently, resources allowing. AFAAS, CCARDESA, NASRO is still in the early stages and not a well-known organization. NASRO, the PAFFO Risk – Modest. Sustainability of FARA A continental umbrella organization on AIS is needed. FARA functioned often as a parallel competitor to institution/forum itself SROs. There are also competitors at the continental level. To be sustainable, FARA must identify its value added, mandate, and ownership, serve as a lean umbrella organization with a clear strategic focus, Risk – High. accountability to its constituency, and a resource mobilization strategy. V. LESSONS AND RECOMMENDATIONS 1. Project Preparation and Supervision Learning by doing vs WB Quality Assurance Process. When the FARA MDTF and the four sister TFs were under preparation, the WB had no guidelines for the TF process or requirements. The TFs served as a pilot for procedures that were later formalized. The design, resource allocation, and M&E were effectively learning-by- doing and subsequently changed many times. The PD would have benefited from internal WB peer review. The Bank team took a significant risk at requesting for additional grant financing for the 2nd restructuring in very uncertain financing conditions. The risks unfolded – and Secretariat and the program activities were continuously streamlined from 2015 onwards, with negative effect on implementation. High Task Team Leader turn-over at the end of the MDTF was not conducive to quality supervision and operational support. Significant delays with disbursement (2017-2018) partly arose from these changes. 2. Monitoring & Evaluation Much greater emphasis on a rigorous M&E was needed at entry and during supervision. The M&E set-up was complex whereas capacity for M&E was limited. Effort was needed to assess the capacity and provide a minimum level of understanding of results-based M&E. The RF was revised twice informally, far before formal approval. Page 29 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Such significant informal changes by FARA should have been nipped at bud and wait until formal restructuring. 3. Sustainability and Ownership The establishment of the FARA MDTF was an important element of coordination and harmonization efforts under CAADP to support the expansion of ARD activities at regional and continental levels. Regional and continental investments into ARD have grown and are supported by a more mature network of actors. Since the closing of the FARA MDTF, donor harmonization efforts have largely ceased. Significant effort by the donors themselves and/or an African coordinating organization is required to make most out of the diverse projects. CAADP program was still more donor than Africa-led, giving rise to commitment and funding challenges. Donors had growing concerns about the absence of exit strategy and realistic and sustainable co-investments from governments and RECs. FARA – as the only CAADP lead organization - has not received any funding from AUC, NEPAD or RECs. Any follow-up support would need to be truly Africa-led. Limited progress with S3A at the country level presented a missed opportunity to drum up African ownership. The development of S3A was a consultative process and an effort to drum up African ownership and leadership for the science agenda. While most countries are committed to integrating S3A into their NAIPs32 the SROs have not yet mainstreamed S3A into their country processes33. Institutional sustainability and resource mobilization considerations should be considered at the onset. The scale of FARA’s activities exposed it to many external variables. Retrospectively, FARA’s size was overestimated (see section III.B) and it became a somewhat isolated large organization with high level salaries and at times superfluous expenses, requiring significant resources to stay functional. FARA became to some extent a victim of the fluctuating donor priorities, both DPs and the constituency questioning ‘value for money’. FARA has not been able to design a sustainable exit strategy from continued donor support. 4. Collaboration Model The original premise was correct – the African continent needs and benefits from having a continental organization dedicated to coordinating IAR4D investments. FARA has contributed to establishment and strengthening of a network of continental, sub-regional and national organizations working on AIS. While the network is far from perfect, it did not exist before. FARA has also significantly raised the visibility of African IAR4D organizations and their needs at the continental and global arenas. The choice of PDOs and RF did limited justice for the interventions and overall achievements (in Annex 9) during the FARA MDTF. FARA has yet to live up to its potential and finds its mandate and collaboration model questioned. FARA’s leadership has not always been strategic but opportunistic, giving rise to ‘lost’ mandate and mission creep. FARA often worked as an isolated parallel organization rather than a continental apex. Currently, FARA faces increasing competition both at the sub-regional and continental levels. Since FARA’s inception, the number of African ARD- innovation oriented organizations has proliferated, and significant overlap exists among FARA, AGRA, FANRPAN, 32 Commitment varies from limited to good, some countries with signed agreements and ear-marked resource allocation. 33Resource leveraging efforts of FARA have resulted in a EC funded SISTA project (engaging the previous Consortium institutions) to be rolled out in July 2019. It aims to bring the S3A mainstreaming to the fore. However SROs and AFAAS are not on board. Page 30 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) AATF and AUDA-NEPAD – either in topics and/or in geographic coverage. Ownership also requires clarification – currently FARA membership is open. Yet FARA has a mandate to serve its constituency. Challenges with subsidiarity and lack of culture of collaboration and trust mire the African AIS. Challenges with subsidiarity (on behalf of all actors, not only FARA) have undermined FARA’s accountability to its constituency and/or constituency’s satisfaction with FARA services (Data in Annex 9). SROs have also built their own capacity, resulting in overlapping responsibilities. Subsidiarity is however not static - any future action requires finding common understanding and agreement on subsidiarity and how it is applied in different programs. FARA has not been able to create a culture of collaboration and trust among the SROs. SROs have not had interest in funding continental activities nor aligning their work with FARA without ear-marked funding. While SROs focus on working with their NARS constituency, they rarely collaborate with other SROs. Most organizations are also competing for the same limited resources - the competitive considerations frequently overrunning collaborative considerations. Collaboration with Northern and Southern Partners presented an underutilized opportunity. FARA facilitated several N-S and S-S partnerships over the years, one of the most notable ones being the Africa-Brazil Marketplace. Despite the achievements associated with the Africa-Brazil Marketplace (see Annex 4), feedback points to limited progress with this activity that has tremendous potential for a continent starved of resources for ARD-innovation. Significant effort has already been placed on re-defining the role and functions of a Continental Organization. The institutional analysis verified the need for a continental apex coordinator. SROs, AFAAS and SROs together developed a follow-up proposal on a continental Consortium, to be led by FARA, that was found to be in a best position to lead the Consortium. The FARA Board did however not approve it, mostly because of dissatisfaction expressed by SROs. It is obvious that competitive pressures and lack of trust prevent the African continent to live up to its own potential in this respect. The recommendation to FARA is to critically review the findings, assess accountability-trust and leadership challenges and develop a cost-effective approach to serving as a true continental apex organization. Such organization must have African ownership, reflected not only in name but also in finance, namely from African governments and the private sector. Current donor dependency subjects FARA to changing donor priorities and undermines accountability to the constituency. Such an Apex organization would need to focus on a continental mandate and on activities that are strategic and add value to what the constituency at the regional and national levels are currently doing well. Apex must build upon its achievements. The key partners, SROs, have grown in number and have strengthened capacity and confidence to re-define their own mandates and comparative advantages in contributing to ARD-innovation. Significant emphasis is needed on building trust and FARA serving as a coordinator, collaborator and catalyzer of collective action. Without trust and coordination, the apex will function in isolation and unable to harness synergies. The apex organization should be lean and nimble member-based organization, such as a Forum, a Community of Practice or a Consortium with limited personnel and fixed costs and reciprocal governance representation. SROs, AFAAS, NARIs, Universities, private sector along with other on-the-ground level actors, can be directly responsible for implementation. A lean organization is also better able to attract funding. New funding strategy should go hand in hand with evidence of relevance, sustainability and effectiveness. Different organizational options exist Page 31 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) (Annex 7). Lessons from CoPs are also useful34. 5. Linkage with other World Bank projects Lack of coordination with WB supported regional agricultural projects. FARA played a significant role in contributing to the development of WB financed WAPP, EAPP, and APPSA. FARA’s engagement in these programs was diminished given its continental role, and SROs took the lead. FARA had no systematic role in coordination and sharing of experiences, nor were the projects properly aligned e.g. with the M&E of FARA MDTF. The projects were not well coordinated among themselves either, pointing to missed opportunities affecting WB, SROs and relevant countries. The MDTFs targeting SROs and AFAAS would have also benefited from coordination, such as joint planning events, aligned reporting and M&E, which could have improved resource use efficiency and . exchange of lessons learned on procedures. 34Gilbert Probst and Stefano Borzillo 2008. Why communities of practice succeed and why they fail. European Management Journal, October 2008, Pages 335-347. Page 32 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Africa's capacity for agricultural innovation is strengthened Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - PDO 1) Increase in Number 0.00 2031491.00 1790395.00 number of individuals, groups, organizations directly 01-Jan-2014 01-Jan-2014 31-Dec-2018 affected or reached by FARA interventions. Farmers Number 1251000.00 1589175.00 1786559.00 01-Jan-2014 01-Jan-2014 31-Dec-2018 Professionals and Number 88971.00 244876.00 201179.00 Practitioners 01-Jan-2014 01-Jan-2014 31-Dec-2018 Comments (achievements against targets): All Farmers – 1,589,175 (target 1,786,559) Page 33 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Male: target 1,196,995 and achievement 1,096,624 Female: target 589,564 and achievement 500,633 Professionals & practitioners – 201,179 (target 244,876). Male: target 164,067 and achievement 139,191 Female: target 80,809 and achievement 67,646 Countries: 41 against target 56 Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - PDO 2) Increase in Percentage 14300.00 90.00 69.00 core competencies, capabilities and capacities for 01-Jan-2014 01-Jan-2014 31-Dec-2017 innovation among targeted ARD actors (cumulative improvement) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - PDO 3) Degree of Percentage 67.00 10.00 48.80 Page 34 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) stakeholder satisfaction with 01-Jan-2014 01-Jan-2014 31-Dec-2017 FARA performance and quality of products and services (10% annual increment) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - PDO 4) Level of Percentage 0.00 2.50 2.35 annual contributions by African Governments and 01-Jan-2014 01-Jan-2014 31-Dec-2017 institutions to agricultural research funding. Agriculture as a % of public Percentage 4.00 10.00 4.40 expenditure 01-Jan-2014 01-Jan-2014 31-Dec-2017 Agricultural research Percentage 0.51 1.00 0.46 intensity ration (ARI) 01-Jan-2014 01-Jan-2014 31-Dec-2017 % of FARA Revenue Percentage 0.00 60.00 146.00 01-Jan-2014 01-Jan-2014 31-Dec-2018 Comments (achievements against targets): Page 35 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) A.2 Intermediate Results Indicators Component: Strategic Priorities Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KRI 1.2) Functional Number 133.00 1000.00 859.00 partnershiops and platforms for agricultural innovation 01-Jan-2014 01-Jan-2014 31-Dec-2017 and trade among African stakeholders and between them and northern and southern partners established. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR1.1) Countries Number 0.00 15.00 41.00 with AR&D agendas being influenced by the outcome of 01-Jan-2014 01-Jan-2014 31-Dec-2017 foresight studies. Comments (achievements against targets): Page 36 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR2.1) Institutions Number 0.00 300.00 254.00 adopting FARA-initiated interventions or mechanisms 01-Jan-2014 01-Jan-2014 31-Dec-2018 for identifying, articulating and/or addressing capacity needs. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR1.3) Participants Number 0.00 24265796.00 17552370.00 (directly) reached, participating or contributing 01-Jan-2014 01-Jan-2014 31-Dec-2017 to innovation platforms, consultations, workshops, meetings. Farmers directly Number 1250000.00 1786559.00 1515500.00 participating 01-Jan-2014 01-Jan-2014 31-Dec-2017 Farmers indirectly Number 15300000.00 22400000.00 15900000.00 benefitting 01-Jan-2014 01-Jan-2014 31-Dec-2017 Page 37 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Participants in meetings Number 1650.00 8800.00 9062.00 and workshops 01-Jan-2014 01-Jan-2014 31-Dec-2017 Professionals trained Number 4731.00 244876.00 201179.00 01-Jan-2014 01-Jan-2014 31-Dec-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR2.2) Institutions Number 0.00 150.00 136.00 (disaggregated by category) whose capacity development 01-Jan-2014 01-Jan-2014 31-Dec-2018 needs have been assessed and/or supported. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR2.3) Functional Number 2.00 3.00 4.00 communities of Practice for creating gender-sensitive 01-Jan-2014 01-Jan-2014 31-Dec-2018 capacities formed and addressing identified capacity deficits in the design Page 38 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) & implemetnation of ARD programs. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR3.4) Platforms Number 0.00 373.00 120.00 used for information delivery and exchange 01-Jan-2014 01-Jan-2014 31-Dec-2017 eRails Number 54.00 200.00 120.00 01-Jan-2014 01-Jan-2014 31-Dec-2017 IPTAs Number 145.00 173.00 120.00 01-Jan-2014 01-Jan-2014 31-Dec-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR3.1) Number of Number 32.00 52.00 44.00 countries and RECs in which FAAP principles and 01-Jan-2014 01-Jan-2014 31-Dec-2017 guidelines have been embedded in CAADP national Page 39 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) and/or regional compacts and AFSIPs Volume of technical Number 1500.00 2138.00 2649.00 information deposited on the website and social 01-Jan-2014 01-Jan-2014 31-Dec-2018 media pages Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR3.2) Number 0.00 2350.00 2839.00 Information and knowledge products/packages (briefs, 01-Jan-2014 01-Jan-2014 31-Dec-2017 reports, scientific papers and publications documentaries) produced and made available to stakeholders (10% annual increment) Books, manuals and Number 63.00 104.00 103.00 technical reports 01-Jan-2014 01-Jan-2014 31-Dec-2017 Staff publications - journals, Number 33.00 108.00 87.00 articles, conference papers 01-Jan-2014 01-Jan-2014 31-Dec-2017 Comments (achievements against targets): Page 40 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR3.3) People Amount(USD) 82889.00 539214.00 639437.00 reached with information through continental 01-Jan-2014 01-Jan-2014 31-Dec-2018 information and knowledge sharing platforms (websites, publications, visual and social media) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR3.5) Degree of Number 11.00 34.00 41.00 improvement in availability of ICTs amongst targeted 01-Jan-2014 01-Jan-2014 31-Dec-2017 NARS institutions Comments (achievements against targets): Component: Corporate Governance Reform Stream and Program Management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 41 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) (MTOP II - KR4.1) Pluralistic Text N/A N/A Yes decision making processes (stakeholder representation 01-Jan-2014 01-Jan-2014 31-Dec-2017 in governance bodies, program management) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR4.2) Compliance Text Operational Manual N/A Yes with operational procedures and Governance and standards Manual 01-Jan-2014 01-Jan-2014 31-Dec-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion (MTOP II - KR4.3) Annual Number 1000000.00 1610000.00 2350000.00 growth of FARA reserve fund 01-Jan-2014 01-Jan-2014 31-Dec-2017 Comments (achievements against targets): Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 42 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Target Completion (MTOP II - KR4.4) AWP Percentage 96.00 100.00 63.00 activities implemented on target, on budget and time 01-Jan-2014 01-Jan-2014 31-Dec-2017 Comments (achievements against targets): Page 43 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) A. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 (MDTF) -- To align African agricultural institutions at the national, regional and continental levels with CAADP Pillar IV FAAP Principles for effective research, extension, and training and education 1. Number of FAAP compliant CAADP country compacts addressing agricultural research, advisory services, and higher education Outcome Indicators 2. Number of FAAP compliant regional CAADP compacts addressing agricultural research, advisory services, and higher education signed. 3. % of AWP programs/ activities implemented on target, on budget and time 1. NSF 1. Number of CIPs that integrate CAADP Pillar IV elements (agricultural research, advisory services, and education) 2. NSF 2(a). Number of gender-sensitive continental platforms on knowledge exchange and technology dissemination for ARD in Africa established. 3. NSF 2(b). Number of stakeholders using gender-sensitive knowledge management tools for knowledge exchange at continental level 4. NSF 3(a). Number of briefs on strategic analysis of policy and market issues produced Intermediate Results Indicators 5. NSF 3(b). Number of policy recommendations synthesized from policy dialogues. 6. NSF 4. Number of capacity strengthening initiatives developed & implemented 7. NSF 4(b). % targeted organisations showing improved performance. 8. NSF 5(a). Number of functional innovation platforms established. 9. NSF 5(b). Number of institutions (SROs, NARS, NARS, private sector, farmer organisations) that have adopted the innovation platform approach. 10. NSF 5(c). Number of partnerships and Strategic Alliances established. Component 1. Networking Support Functions (NSFs) Inputs to Science week and Science Agenda, and partnership 1. Foresighting studies completed and shared amongst countries Key Outputs by Component 2. AUC framework and other decisions making processes supported (linked to the achievement of the 3. Science agenda and consortium advanced Objective/Outcome 1) 4. Partnership between N-S and S-S facilitated 5. Innovation platform approach advocated for in diverse projects Capacity-building Page 44 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) 6. AFAAS, CCARDESA, NASRO, PAFFO and TEAM Africa and others with understanding of capacity needs 7. Organizations provided with TA ad support capacity-building on diverse topics 8. CoPs established pm e.g. for partnerships on capacity, with private sector, graduate students Engagement with RECs and countries 9. ICT speed and capacity performance, reliability of internet access, equity, service quality, cost effectiveness issues in organizations addressed. Component 2. Corporate Governance Reform Stream and Program Management 10. FARA board and GA with adequate representation and skill set 11. Secretariat with capable staff 12. Resource mobilization effort 13. FARA operations implemented on target, budget and time. Objective/Outcome 2 (MTOP2) -- To strengthen Africa’s capacity for agricultural innovation with purposefully determined outcomes, creating capacity for innovation and enabling environments for implementation’ 1. Percentage increase in number of individuals, groups, organizations directly affected or reached by FARA interventions (disaggregated by gender) 2. Percentage increase in core competencies, capabilities and capacities for innovation among targeted Outcome Indicators (individual, organizational/inter-organizational and/or institutional) ARD actors 3. Degree of stakeholder satisfaction with FARA performance and quality of products and services 4. Level of annual contributions by African Governments and institutions to FARA funding (as % of FARA revenue) 1. Countries with AR&D agendas being influenced by the outcome of foresight studies 2. Functional partnerships and platforms for agricultural innovation and trade among African stakeholders and between them and northern and southern partners established 3. Participants (directly) reached, participating or contributing to innovation platforms, consultations, workshops, meetings. Intermediate Results Indicators 4. No of institutions adopting FARA-initiated interventions or mechanisms for identifying, articulating and/or addressing capacity needs 5. No of institutions (disaggregated by category) whose capacity development needs have been assessed and/or supported (enhanced knowledge, skills and attitudes of individuals delivered through training workshops; changes in organizational design and culture, accountability, responsiveness, transparency and efficiency) Page 45 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) 6. No of functional Communities of Practice for creating gender sensitive capacities formed and addressing identified capacity deficits in the design & implementation of ARD programs 7. No of countries and RECs in which FAAP principles and guidelines have been embedded in CAADP national and/or regional compacts and AFSIPs 8. No of information and knowledge products/packages (briefs, reports, scientific papers and publications documentaries) produced and made available to stakeholders 9. No of people reached with information through continental information and knowledge sharing platforms (websites, publications, visual and social media) 10. No of platforms used for information delivery and exchange 11. Degree of improvement in availability of ICTs (magnitude of ICT speed and capacity performance, reliability of internet access, equity, service quality, cost-effectiveness) amongst targeted NARS institutions 12. Pluralistic decision-making processes (stakeholder representation in governance bodies, program management) 13. Compliance with operational procedures and standards 14. Annual growth rate of FARA reserve fund 15. AWP activities implemented on target, on budget, and time Key Outputs by Component As under Objective Outcome 1 (linked to the achievement of the Objective/Outcome 2) Page 46 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) FARA MDTF RESULTS FRAMEWORK – PRE-RESTRUCTURING (2009-2013) PDO To align African agricultural institutions at the national, regional and continental levels with CAADP Pillar IV FAAP Principles for effective research, extension, and training and education’ Revised Indicators– status in Nov 2013 Discussion of results and achievement (indicators & interview findings) Number of FAAP compliant CAADP country compacts This indicator may reflect the change in language in official documents, and in the best of circumstances, a addressing agricultural research, advisory services, change in beneficiaries’ (e.g., NARIs, advisory services) understanding and behavior. Signing indicated and higher education country’s commitment to CAADP process. The FARA MDTF was reasonably good (81%) at achieving this outcome. Target: 37 country compacts Achievement: 81 % (30 compacts) Surveys and interviews verify this finding - countries were supported by FARA on developing CAADP aligned strategies and investment plans. NARS benefited also from wide-spread access to knowledge, technology, briefs (with 70% use rate) and participated in planning and priority setting processes along with IP – that contributed to awareness and need for integration of CAADP into country programs. Number of FAAP compliant regional CAADP compacts Indicator reflects the change in language in official documents and/or commitment to CAADP. Achievement addressing agricultural research, advisory services, rate of 33% could be inferred to reflect poor adoption of FAAP by the RECs. However, the indicator target was and higher education signed. set rather independent of stakeholders’ views, and simply reflected the total number of RECs in Africa. Low capacity and unwillingness of RECs to engage in CAADP process limited achievements. Target: 6 regional compacts Achievement: 33% (2 compacts) Studies and interviews indicate that SROs were aligned with FAAP and widely promoted capacity-building activities (FARA TBAs), and adoption of IAR4D approaches. % of AWP programs/ activities implemented on This indicator is a poor choice to measure achievement of outcomes. While achievement rate is 118%, it does target, on budget and time not provide evidence on the achievement of outcomes. Target: 80 defined activities; Achievement: 118% Intermediate Results Baseline Target Achievement Assessment of Indicator and Conclusion Networking Functions – Intermediate Indicator and rate (%) Strengthened institutional and organizational structures/arrangements for regional ARD # CIPs that integrate CAADP Pillar IV elements 0 25 33 (132) The same as at PDO level, reflecting CAADP aligned changes in country (research, advisory services, and education) implementation plans. Target achieved at 132%. Enhanced access to knowledge and technology for agricultural stakeholders in Africa # gender-sensitive continental platforms on 3 6 6 (100) Reflects access to knowledge and information, not necessarily use of. Target knowledge exchange and technology dissemination achieved 100%. for ARD in Africa established. # innovations facilitated through technology 0 12 11 (92) Refers to technology-innovation shared (but not necessarily used) among AIS dissemination platforms actors. Targets almost met. # national information and learning systems on 0 25 29 (116) Points that countries value the need to share relevant information-learning with agricultural innovations established other AIS actors. Targets exceeded. # stakeholders using gender-sensitive knowledge 200 2500 2821 (113) Reflects access to knowledge and information, not necessarily use of. Numbers management tools for knowledge exchange at are based on membership in FARA online network, IPTAs, RAILS and AfricaAdapt. continental level Targets exceeded. Page 47 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) # country platforms integrated into continental 0 25 48 (192) Refers to connected country-region-continental information system. Targets far platforms exceeded. Strengthened decision-making options for policy, institutions and markets # briefs on strategic analysis of policy and market 3 33 19 (58) Reflects access to knowledge and information, not necessarily use of. Target issues produced modestly achieved. # recipients making use of policy briefs - 60 - No data – however previous studies indicate that 70% of actors use policy and other analysis. # policy recommendations synthesized from policy 0 60 81 (135) Reflects access to knowledge and information, not necessarily use of. Target dialogues. exceeded. # countries making use of the recommendations - 15 - Important indicator without data. However previous studies indicate that 70% of synthesized from policy dialogues actors use policy and other analysis. Strengthened human, organizational and institutional capacity for agricultural innovation in Africa # capacity strengthening initiatives developed & 2 7 7 (100) Many bilaterally funded activities e.g. TAP initiative, and EC funded initiative on implemented “Human and Institutional Capacity Re quirements for Sustainable Food Security in African Countries” # funded capacity strengthening initiatives 2 5 6 (120) Refers mainly to bilaterally funded capacity-building initiatives (see section on implemented Factors affecting implementation) % targeted organizations with improved 0 80 71 % 5 of 7 surveyed institutes registered overall positive changes in the SWOT factors organizational performance (a composite index i.e 71%. incorporating changes observed in SWOT factors) # agricultural innovations developed by targeted 0 4 3 (75) Reflects change in capacity, resources and/or adoption of new IAR4D African NARS approaches. Limitations exist - achievement rate 75%. % women scientists in targeted NARS 22% 40 30-41% Reflects advocacy for gender in education, hiring and promotion. Targets mostly met. Improved support for agricultural innovation platforms (IP) in Africa # functional IPs established. 0 51 66 (129) A good indicator reflecting change in attitudes, capacity and AIS in practice. Country demand for these exceeded targets. # institutions (SROs, NARS, NARS, private sector, 0 20 27 (135) Reflects change in attitudes, capacity and AIS in practice. Target exceeded. FOs) that have adopted the IP approach. # operational guides for establishing IP developed 0 8 14 (175) Refers to development of good practices and provision of evidence. Targets exceeded. # IPs (SROs, NARS, NGOs, private sector, farmers’ 0 51 66 (129) Reflects use of good practice guidelines (of FARA) – targets exceeded. organisations) that are using the operational guides # partnerships and Strategic Alliances established. 2 8 11 (138) Reflects regional and global knowledge and capacity exchange. Target exceeded. Page 48 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) FARA MDTF RESULTS FRAMEWORK – POST-RESTRUCTURING (2014-2018) PDO To strengthen Africa’s capacity for agricultural innovation with purposefully determined outcomes, creating capacity for innovation and enabling environments for implementation Indicators Discussion of results and achievement Percentage increase in number of individuals, groups, organizations Achievement: aggregate average for farmers and professionals 86% directly affected or reached by FARA interventions (disaggregated All farmers: 89%, all professionals: 82% and countries: 73% by gender) This indicator is a semi-good indicator with a potential to reflect a change in behavior. The All Farmers – 1,589,175 (target 1,786,559) underlying assumption is that FARA services are relevant and new learning is internalized. Based Male: target 1,196,995 and achievement 1,096,624 on the aggregate achievement rate of 86% (Dec 2018 numbers), FARA interventions reached the Female: target 589,564 and achievement 500,633 beneficiaries. Professionals & practitioners – 201,179 (target 244,876). Male: target 164,067 and achievement 139,191 Based on the studies and interviews, both NARS and SROs benefited from participation in Female: target 80,809 and achievement 67,646 planning, prioritization and scientific events, capacity-building, CAADP-FAAP-S3A processes as Countries: 41 against target 56 well as key IAR4D approaches such as IPs. Percentage increase in core competencies, capabilities and Achievement rate: 76.6% (organizational capacity) and 86.5% (human capital) and at the capacities for innovation among targeted (individual, aggregate level 81.5% organizational/inter-organizational and/or institutional) ARD actors This indicator provides important qualitative data to assess change in capacity. The indices measure. Based on the information available, the project nearly reached the targets in Percentage change in Organizational capacity/indices: target > 90, organizational and human capacity among target groups. achievement 69. Change in human capital indices – 50% growth in 10 years Based on the interviews, NARIs have appreciated capacity-building activities, those covered by Target 16,750, achievement 14,500. the FARA MDTF and especially the TBAs financed by International Partners (implemented by FARA or SROs). Degree of stakeholder satisfaction with FARA performance and This indicator is a good measure of FARA’s relevance, reach and accountability. quality of products and services (10% annual increment) Baseline with the satisfaction rate was 67%. As of December 2017, the satisfaction rate is ‘negative’ given it had dropped to 48.8%. No M&E data is available from 2017 onwards. Target: 10% annual increment, 100%. Based on the achievement %, the project did not meet appropriate satisfaction level. Interviews Achievement: 48.8 % vs Fair-Good and surveys partly verify this finding – SROs consider FARA often working in isolation and not adhering to subsidiarity. The views may be subject to misunderstanding. However, satisfaction with FARA provided services varies – for example overall ratings vary from Fair to Good (mostly) whereas satisfaction with IP approach and increased visibility is High. Level of annual contributions by African Governments and FARA was to maintain a 100 days expenditure at any point in time. institutions to FARA funding (as % of FARA revenue) Based on the achievement rate, the project mostly met its targets. Page 49 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Target: 2.5% Achievement: 2.35 % Intermediate Results - Strategic Priorities Baseline Target AchievementAchievement Comments – assessment Rate (%) (IR data used is from December 2017) African agricultural stakeholders determining how the sector should be transformed and establishing the needed collective actions in a gender sensitive manner** # countries with AR&D agendas being 0 15 41 273 Reflects access to and use of relevant data for decision- influenced by foresight studies making. # functional partnerships and platforms for 133 1000 859 86 Good indicator on collective action, reflects adoption of AIS agricultural innovation and trade among (previously in practice. Targets nearly met. Note – target level raised African stakeholders and between them and 214) substantially. northern and southern partners established # participants (directly) reached, participating Farmers directly - Measures reach & participation but not necessarily benefits. or contributing to IPs, consultations, 1,250,000 1,786,559 1,589,175 89 Attribution is a challenge. workshops, meetings. Farmers indirectly - 22,400,000 15,900,000 71 15,300,000 Participants in 8,800 9,062 103 meetings - 1,650 Professionals 70,437 54,133 77 trained - 4,731 Strengthened and integrated continental capacity responding to stakeholder demands within the agricultural innovation system in a gender sensitive manner # institutions adopting FARA-initiated 0 300 254 85 A good qualitative indicator for changed practices within an interventions or mechanisms for identifying, organization. Targets nearly met. articulating and/or addressing capacity needs # institutions w hose capacity development 0 150 136 91 A relatively good indicator on capacity improvement albeit needs have been assessed and/or needs assessment alone or support do not necessarily supported35 reflect a difference in capacity. Targets nearly met. # functional CoPs for creating gender 2 3 4 133 Reflects behavior changes (a drive for capacity sensitive capacities formed and addressing improvements). CoPs -foresight, mechanization, innovation identified capacity deficits in the design & systems and biomassweb. Targets exceeded. implementation of ARD programs Enabling environment for increased AR4D investment and implementation of agricultural innovation systems in a gender sensitive manner # countries and RECs in which FAAP principles 32 52 44 85 A reasonable indicator however it provides only indirect and guidelines have been embedded in measure of behavior change. Targets nearly met or CAADP national and/or regional compacts exceeded. and AFSIPs # information and knowledge Books, manuals, 104 103 99 Measures access to – only a proxy to use of knowledge. products/packages (briefs, reports, scientific reports – 63 papers and publications documentaries) Staff publications – 108 87 80.5 produced and made available to stakeholders 33 Volume of technical 35 Enhanced knowledge, skills and attitudes of individuals delivered through training workshops; changes in organizational design and culture, accountability, responsiveness, transparency and efficiency Page 50 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) information deposited on the website and social media pages - 1500 2,138 2,649 124 # people reached with information through 82,889 539,214 639,437 119 A measure of access, but only an indirect measure of continental information and knowledge change. Targets exceeded. sharing platforms (websites, publications, visual and social media) # platforms used for information delivery and eRAILS 200 120 60 A good indicator capturing use. exchange IPTAs - 145 173 -- -- Degree of improvement in availability of ICTs 11 34 41 120 A good qualitative indicator of ICT improvements within (magnitude of ICT speed and capacity organizations. Target exceeded. performance, reliability of internet access, equity, service quality, cost-effectiveness) amongst targeted NARS institutions Strengthened gender responsive governance and management systems – component 2 Pluralistic decision-making processes N/A Yes A good indicator of use. (stakeholder representation in governance bodies, program management) Compliance with operational procedures and Operational and Yes standards governance manual Annual growth rate of FARA reserve fund 1,000,000 1,610,000 2,350,000 146 An indirect measure of accountability and satisfaction. AWP activities implemented on target, on 96 100 63 63 Poor choice. budget, and time Page 51 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Azeb Fissha Mekonnen, Abdoulaye Toure Task Team Leader(s) Charles John Aryee Ashong Procurement Specialist(s) Robert Wallace DeGraft-Hanson Financial Management Specialist Tekola Dejene Team Member Lydia Sam Team Member Wilhelmus Gerardus Janssen Window Manager Bremala Malli Team Member Abdelaziz Lagnaoui Environmental Specialist Riikka Rajalahti Team Member Kofi Nouve Team Member Alidu Babatu Adam Social Specialist Page 52 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) A. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY09 26.382 93,544.41 FY10 20.800 56,921.00 Total 47.18 150,465.41 Supervision/ICR FY10 33.000 129,110.46 FY11 65.409 406,767.14 FY12 33.171 194,925.01 FY13 21.652 136,400.07 FY14 27.947 270,936.24 FY15 22.110 206,387.59 FY16 19.999 266,528.58 FY17 16.101 117,148.34 FY18 15.890 115,279.47 FY19 21.138 137,914.11 Total 276.42 1,981,397.01 Page 53 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Strategic Priorities 0 12.10 0 Corporate Governance Reform Stream and Program 0 12.70 0 Management Salaries and Benefits 0 18.50 0 Sub-Grants for Sub-Projects 0 1.90 0 Total 0.00 45.20 0.00 Page 54 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 4. EFFICIENCY ANALYSIS Due to the nature of the FARA MDTF project, traditional rate of return calculation is not appropriate. The below tables (Resource Allocation, Bilateral Projects & Funding, continental ARD umbrella organizations and Examples of expenditure use support the analysis and discussion in section on Efficiency. Resource allocation per component US$ and % of total – core MDTF funds from 2009-2018. Original budget Disbursed & Additional Total Committed Actual Disbursed Component 2008 (US$M) Committed Funds Financing 2013 Financing (US$M) April 2018 (%) (US$M) (%) (US$M) (%) (%) (US$M) (%) Networking Support Functions 10.43 (23.5) 8.79 (27.5) 11.66 (26.8) 20.45 (27.1) Strategic Priorities 12.10 (26.8) Corporate Governance Reform 21.57 (48.7) 21.88 (68.4) 29.08 (66.9) 50.96 (67.6) Stream and Program Management 31.20 (69) Sub-Grants for Sub-Projects 12.30 (27.8) 1.3 (4.1) 2.72 (6.3) 4.02 (5.3) 1.90 (4.2) Total Program Budget 44.30 31.97 43.46 75.43 45.20 Management and Supervision of the 1.60 1.58 1.53 3.11 MDTF (Bank-executed) 1.90 Total with WB Management 45.90 33.55 45.00 78.55 47.10 1. Bilateral Projects and Funding throughout the FARA MDTF. Grant Amount (USD) Project Name Funding Partner EC/DFID/Italy/ Netherlands/The Sub-Saharan Challenge Programme (with Innovation Platforms) 35,708,600 World Bank CRP1.2 Humid Tropics (CGIAR Research Program) CGIAR/IITA 5,012,590 SCARDA - Strengthening Capacity for Agricultural Research & DfID, UK 10,380,396 Development in Africa program PSTAD – The Promotion of Science and Technology for Agricultural African Development Bank (AfDB) 21,948,417 Development in SADC Region PAEPARD – The Platform for African – European Partnership in European Commission 13,610,478 Agricultural Research for Development SABIMA - Sub-Saharan Africa’s capacity for Safe Biotechnology Syngenta Foundation for 1,391,046 Management Sustainable Agriculture UniBRAIN - Universities, Business and Research in Agricultural DANIDA 20,406,826 Innovation PARI - Program of Accompanying Research for Agricultural Innovation BMZ/ZEF 2,881,716 Africa Adapt - Bilingual network (French/English) to facilitate the flow of climate change adaptation knowledge for sustainable livelihoods IDRC/IDS 343,884 between researchers, policy makers, civil society organisations and communities Africa Brazil Agric Innovation Marketplace 36 Various through EMBRAPA 825,360 36 The Africa Brazil Innovation market place had many achievements along the research and technology delivery pathway. A total of 76 research projects (600 proposals received) were financed covering 13 African countries and eight LAC countries. 1,117 experts were trained on diverse topics, 34 projects were completed, along with 93 events and 131 publications. 147 technologies and products were developed; 924 germplasm Page 55 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Climate Smart Agriculture (CSA) NORAD, Norway 882,029 Biomassweb - Improving food security in Africa through increased BMZ/ZEF 521,928 system productivity of biomass-based value webs TIGA - Ex-ante technology assessment for inclusive poverty reduction BMZ/ZEF 84,500 and sustainable productivity growth in agriculture TAP – Tropical Agriculture Platform (AIS capacity-building) FAO 60,682 BNFB - The Building Nutritious Food Baskets - Project BMGF/CIP 186,426 YPARD - Young Professionals for Agricultural Development GFAR 20,500 ECoSIB - Entrepreneurship for Commercial Seed Incubation Business USAID/AGRA 449,161 ProintenseAfrica - Pathways to sustainable intensification of the agro- European Commission 66,418 food system in Africa TAAT - Technologies for African Agricultural Transformation AfDB 2,189,000 116,969,956 2. Examples of continental ARD umbrella organizations - with different membership numbers and capacities. Organization Activities Deliverables Secretariat Governance Foragro – A Forum To promote dialogue, actively Debate Forum; 2 persons Steering Committee and www.foragro.org discuss needs and opportunities, Information and knowledge on Assembly of Members; generate analysis and proposals and priority topics; Secretariat foster partnerships Platform for coordinating action among stakeholders; Platform for facilitating information & knowledge dissemination. APAARI Knowledge management, Deliverables are mainly workshop and Eight member staff The General Assembly, http://www.apaari.org Partnerships, Capacity, Advocacy symposium documents on priority supported by meeting once every two topics and knowledge management. consultants years; Secretariat SCAR in Europe - Standing Enhances cooperation, coordination, Delivers through various working Secretariat is also a Plenary Meeting (twice a Committee on Agricultural and information exchange between groups, each with its own working group year, expenses by EC Research the Member States on strategic responsibilities but, ultimately, (member countries Commission) and a https://scar-europe.org/ policy advice, foresight process, reporting to the six-monthly plenary delegate them) and Secretariat common research areas, mapping meeting of official delegates meets regularly to SCAR research resources. plan activities. 3. Examples of resource use efficiency (or lack thereof): - The WB initiated in-depth fiduciary and value for money review for expenditure (2016) for years 2012-2014 made a few observations on poor value for money practices with the EDs office (such as excessive travel, daily subsistence, accommodation costs, purchase of gifts and furniture, use of credit card, etc). Similarly, feedback from stakeholders brought to the fore a sense of entitlement – illustrated by high salaries, expensive trips and cars and lavish celebrations (e.g. FARA at 15, held in South Africa). - Lack of cost efficiency contributed to limited DP commitment. DfID and CIDA withdrew their funding. The Bank’s payment obligation to FARA was limited by the amount of funds made available by its donors. The EC’s were exchanged between Brazil and Africa and 81 specific knowledge were generated. Female participation rate was 26%. Page 56 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) payment schedule was included in the legal agreement between the Bank and donor and included three payments. The EC’s first payment (50% of the total) was received in June, 2014 and a request for the second payment (45%) was submitted soon afterwards, in January 2015 due to accelerated expenditure of the initial payment. However, the EC was not prepared to make a second payment, so soon after making the first payment, especially given that its financing was to last for a period of 5 years. Instead, a partial payment of €3 million was made in August, 2015. As of May 2016, the total amount disbursed to FARA is represented a disbursement ratio of 93.8%. - Science Week was a successful activity, drawing hundreds of participants from Africa and elsewhere. For the 2013 Accra event FARA paid all participant costs, including air travel and accommodation (the total costs US$1.18 million). Due to fiduciary conditions, Bank’s views and FARA’s Board’s decision, the 2016 Kigali event was participant paid, the costs totaling only about US$100,000. The event – managed now cost-effectively - still attracted more than 1,200 participants. Page 57 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Key stakeholders – NARS, SROs, AUC, NEPAD, CGIAR, FAO, AFAAS, AAIN - along with WB staff were approached and their responses have been integrated into the report. Page 58 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) Project Documents: Aide Memoires of Implementation Support Missions, 2009 to 2018. Implementation Status & Results Reports, 2009 to 2018. FARA MDTF Project Document – December 1, 2008 (Draft). FARA MDTF Project Appraisal Document (Additional Financing) – December 13, 2013. FARA MDTF Restructuring Paper - June 28, 2012 FARA MDTF Restructuring Paper – 2017 (Draft, not approved) FARA MDTF Mid-Term Reviews 2012 and 2016. FARA Self-Assessment 2012 and 2016. Other Supporting Documents: Agricultural Innovation Platforms: Framework for Improving Sustainable Livelihoods in Africa. 2016. By Fatunbi A.Oluwole, Anthony Youdeowei, Samuel Ifidon Ohiomoba and Adekunle Adewale, Akinbanijo Yemi. https://faraafrica.org/wp-content/uploads/2017/12/Agricultural-Innovation-Platforms.pdf Experiences and Lessons Learned from the UniBRAIN Agribusiness Incubation Programme. By Carsten Nico Hjortsø Ian Keith Alexander Roberto R. Hernandez Chea. IFRO Report 266. 2017. FAAP 2006. The Framework for African Agricultural Productivity (FAAP). http://www.fara- africa.org/media/uploads/File/FARA%20Publications/FAAP_English.pdf Report on In-depth financial management review and value for money audit of implementation of FARA MDTF. 2016. Forum for Agricultural Research in Africa (FARA). The Science for Agriculture Consortium – Draft Concept Note. September 19-30, 2016. Joint Review (EPMR and MTR) of the Forum for Agricultural Research in Africa (FARA). Final Report January 2012. Science Agenda for Agriculture in Africa. “Connecting Science” to transform agriculture in Africa. FARA 2014. Science for Agriculture in Africa – Analysis of Consortium Partner Institutions. Draft Report. By Genevesi Ogiogio and Ed Rege. May 12, 2017. Strategic Review of DEVCO support to African continental and sub-regional agricultural research organizations: FARA, AFAAS, CORAF/WECARD, ASARECA and CCARDESA. Final Report May 2016. World Bank 2009. World Bank Agriculture Action Plan 2010-2012. World Bank 2011. World Bank Africa Strategy 2011. Africa’s Future and the World Bank’s Support to It. World Bank 2013. World Bank Group Agriculture Action Plan 2013-2015. World Bank 2014. Implementation Completion Report for ASARECA Multi-Donor Trust Fund. 2014. World Bank 2015. Ending Poverty and Hunger by 2030: Agenda for Global Food Systems. World Bank 2017. Implementation Completion Report for CORAF Trust Fund. 2017. Page 59 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 7. FARA’s Background, Mandate and Programs over the years FARA was conceived in the late 1990s by a select group of African scientists and donors to address the continent’s fragmented agricultural research systems. FARA’s predecessor - Special Programme for African Agricultural Research (SPAAR) was formally established in 1997 by the SROs, including CORAF/WECARD, ASARECA and SADC-FANR. SPAAR wassituated within the WB. FARA as it stands today wasformally established in 2002 to function as a continental Forum. FARA, closely affiliated with GFAR, was housed at FAO and had attracted well known technical professionals. In 2006, FARA wasrecommended to shift towardsa more integrated approach in line with the Paris Declaration on Aid Effectivenessand specifically to better harmonize and align the contributions of DPs to FARA. FARA moved its Headquarters from FAO premises to Accra, Ghana. FARA complements the activities of national, international and sub-regional institutions to deliver more responsive and effective services to its stakeholders. It playsadvocacy and coordination rolesfor agriculturalresearch for development (which includes the activitiesof agriculturalresearch per se, but also, inter alia, agricultural advisory services, agricultural training, and education institutions). The demand to mainstream FAAP in CAADP Compacts and the move towards an integrated programme (as opposed to a set of projects) financed by a MDTF became the factors underpinning FARA programming based on five Networking Support Functions corresponding to the five themes in the first MTOP. The 2008-2009 food crisis further alerted African stakeholders and donors alike to pay attention to agricultural research and technology uptake. FARA was one of the beneficiaries of the abundant resources. FARA’s primary role as lead institution working with the RECS, SROs and national institutions is to: (a) employ the FAAP to provide guidance to Pillar IV programs, policies, and institutions; (b) ensure effective integration of the agricultural research, science and technology, extension, and education and training agendas into the CAADP national and regional roundtable process and compact development; (c) facilitate and support the design and implementation of the Pillar IV components of the country and regional compacts; and (d) support the mobilization of resources from internal and external sources for implementation of Pillar IV components and promote adherence with FAAP principles in their implementation. From the MDTF effectiveness(September 2009), the FARA Secretariat had the main responsibility to develop and coordinate a portfolio of continent- wide initiativesin five thematic areas: (i) advocacy of the role of agriculturalresearch; (ii) promotion of functionalpartnershipsand strategic alliances; (iii) stimulating development and dissemination of new technologies and methodologies; (iv) sharing and exchange of information; and (v) capacity strengthening. The overall aim of these initiatives was to enhance the development impact of research. Two principal events during this period marked FARA ’semergence asa significant facilitator of African AR4D: itsrecognition asthe technicalarm of the African Union Commission on matters concerning agricultural research, which subsequently led to its appointment as the lead institution for Pillar IV of CAADP that was concerned with research, technology dissemination and adoption; and (ii) FARA ’s development of a framework through which the challenges prioritized by the CAADP Pillar IV were to be achieved - the Framework for African Agricultural Productivity (FAAP). FARA ’s programme of work during this period was organised around severalcontinent-wide programmes, namely, the sub-Saharan Africa Challenge Programme (SSA CP), SCARDA, DONATA and RAILS. One of the main achievements of this period was the development of an African owned Science Agenda for Agriculture in Africa (2013). Following the restructuring of the MDTF in 2013, the PDO aimed to build on the achievements made during MDTF (MTOP1) with a forward looking thrust based on foresight and strategic analysis. It also sought to widen FARA ’s cohorts of actors that it primarily engaged with (research, education and extension) to include policy and the private sector. Post-restructured project was organised around three SPs: (i) visioning (with foresight and strategic analysis), (ii) integrating and strengthening capacities; (iii) and policy and advocacy. Changes initiated in 2016 (not formally approved by the WB) resulted in further streamlining of activities to match the financial conditions. Whereas in the past FARA focused on creating regional institutions that were required to fill gaps in Africa’s AIS, as these institutions grow in strength and confidence, FARA’s focus will be more on keeping them connected and integrated programmatically in order to avoid debilitating fragmentation of the African AIS. FARA’s new approach is aligned with the transformative phase of the CAADP. FARA is also aware of the need to involve private enterprise more effectively. There is an urgent need for Africa to develop a capacity for forward thinking and proactive preparation for emerging opportunities and threatswith an approach that nurtures the emergence of young talent to play a role in preparing for and shaping the future. FARA strives to ensure that planning for African agricultural development is consistent with Africa’s capacity to deliver and that action is taken to strengthen capacity for more ambitious public and private agricultural development initiatives. This commits the FARA Secretariat to working with research, extension and education institutions and organizations to build and strengthen knowledge, information and skills capacities. The core of the activities is centered on (i) Knowledge Management and Decision Support, (ii) Capacity Development for the Implementation of S3A and (iii) Innovation Systems and Partnerships. Source: PD (2008), PAD (2013), MTR (2016), Aide Memoirs, FARA website. Page 60 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 8. OTHER CHANGES This Annex provides supplemental information on other changes – aligned with section I.B. Key project dates and changes undertaken and parallel influencing reviews and events Project Event Evaluation/Review – Other Influencing Events Approval – Dec 4, 2008 and Effectiveness – Sept 4, 2009 FARA to lead CAADP Pillar IV in 2006, Food Price Crisis 2007-2008 Mid-Term Review (MTR) 1 – Feb 1, 2012 M&E Results Workshop 2011 1st Restructuring – June 28, 2012 Joint Review (External Program Management Review and MTR) – Jan 2012 (PDO Indicators, RF and grant allocation) 2nd Restructuring - Dec 13, 2013 Consultative Development of Science Agenda for Agriculture in Africa (S3A) (PDO, PDO indicators, RF and Components, Additional (in 2013) and formal approval in Jan 2014 – guiding design of MTOP2 Financing and Extension of Closing Date) Streamlining of Secretariat functions and technical programs, Meeting of Heads of State and Malabo Declaration - June 2014 – together from 2015 onwards to match the limited financial resources with financial challenges, triggered informal changes MTR 2 – June 2, 2015 Informal changes by FARA – Jan 2016 (PDO, PDO indicators, RF, components) In-depth Financial Management Review and Value for Money The S3A Consortium Proposal (SROs, AFAAS, FARA) – Sept 2016 (not Audit – May 2016 (for the period of 2015-2015) approved by FARA Board) 3rd Restructuring in 2017 (to formally approve the 2016 WB Initiated Institutional Analysis – May 2017 initiated changes) – dropped Closing – Dec 31, 2018 EC and IFAD support to S3A Implementation in countries - June 2019 1st restructuring (June 28, 2012) – Grant allocation: Funds were reallocated between expenditure categories (Table below) to avoid imbalances across expenditure categories and ensure that FARA can provide the necessary support for relevant ongoing and new work steam. Reallocation of Grant Funds in Restructuring of June 28, 2012. Category of Expenditure Allocation % of Financing Current Revised Current Revised Current Revised Goods, consultants’ services, and No 5,930,785 5,848,877 Such % of Eligible Expenditures as the WB may No Training and Workshops under Part A Change determine for each Annual Program of Activities Change Goods, consultants’ services, Training No 4,253,333 6,934,016 Such % of Eligible Expenditures as the WB may No and Workshops, and Operating Costs Change determine for each Annual program of Activities Change under Part B Salaries and Benefits under Part B No 8,011,961 2,117,107 Such % of Eligible Expenditures as the WB may No Change determine for each Annual Program of Activities Change Sub-grants under Part C No 7,903,921 1,200,000 Such % of Eligible Expenditures as the WB may No Change determine for each Annual Program of Activities Change 26,100,000 16,100,000 2nd restructuring (December 13, 2013) – Revised components: The Components after the 2nd restructuring were as follow: Component 1. Strategic Priorities SP1: Visioning Africa’s agricultural transformation – with foresight, strategic analysis and partnerships. The objective was to enable more productive engagement (new models and analyses) between the public and private sectors, leading to the ‘creation’ of an AIS, trade, investments and marketing. FARA was to support regions to engage in the S3A and to build Africa’s self-reliance in agricultural sciences. FARA were to be more engaged with the AUC’s frameworks – with evidence for decision-making on IAR4D at AU Ministerial Conferences; AU Heads of States and Government Summits; the CAADP Page 61 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Partnership Platform; Africa-European Union Joint Strategy; Africa– India, Africa– Brazil, Africa– China and Africa–Australia and the Africa– Arab research exchange. Additionally, FARA was to promote intra- and inter-continental platforms and partnerships, building on the N-S and S-S partnerships developed over the last ten years. SP2: Integrating capacities for change – by connecting and learning. The objective was to advocate and facilitate strategic reforms, processes and partnerships that coordinate the capacities of ARD actors (countries, institutions, local programs) to enhance capacity strengthening. FARA was to continue to support AFAAS, CCARDESA, NASRO, PAFFO and TEAM Africa, until they become fully established, and foster links between them, other AIS actors, and the specialized agencies of the AU. Entrepreneurs and smallholders-women were to receive special attention in IAR4D processes. SP3: Creating an enabling environment for implementation – by advocating and communicating. The objective was to convince African governments that strong AIS is critical for economic development and food security and must be less donor dependent. FARA were to maintain a knowledge hub to support strategic agricultural policy formulation, promote networking among the economic research community and policy makers, promote the exchange of economic policy research results and the sharing of best practice policy options among policy research institutions, policy makers and the broader community of CAADP stakeholders through several avenues. Component 3. After restructuring, FARA were to provide sub-grants (capacity building, training, networking and advocacy) for specific sub-projects for advisory services, agricultural education and research and development to regional and sub- regional organizations. The component was significantly limited in size given DfID withdrew its support to MDTF. In addition, as of end 2013/early 2014, previously supported AFAAS and CCARDESA secured their own Bank-administered MDTFs with support from the EC. Additional details on themain rationales forthe2nd Restructuring (December13, 2013) and the Planned Restructuring (in 2016) Changing requirements of Africa’s agricultural transformation agenda. The Sixth Joint Annual Meeting of the United NationsEconomic Commission forAfrica(UNECA), the Conference of African Ministers of Finance, Planning and Economic Development and the African Union (AU) Conference of Ministers of Economy and Finance in March 2013 agreed on the need for a refocused program for UNECA that supports the transformation agenda for Africa. The reconfigured programs were to focus on conducting research and normative work to support African member states in transforming their economies from low-income to middle-income status. Widened mandate. The leadership role that FARA played in the implementation of CAADP Pillar IV widened the scope set by this original mandate (which focused on agricultural research), to embrace legitimate involvement with actors spanning the whole agricultural innovation system. The development of Science Agenda for Agriculture in Africa (S3A) specifically guided the role of FARA in the future. The AUC had renewed its Memorandum of Understanding (MoU) with FARA, according to which FARA would continue to provide IAR4D technical assistance to the AUC and its technical agencies, including NPCA, while sustaining the CAADP momentum. Science Agenda for Agriculture in Africa . Since its approvalin 2014, the S3A hasbeen the overarching strategic framework to guide the broad areas of science that have to be developed by the African countries, their stakeholders and partnerships. The strategic thrusts of the S3A in the short to medium term are: the implementation of CAADP; increase domestic public and private sector investment; creating the enabling environment for sustainable application of science for agriculture; and to double current level of Agricultural Total Factor Productivity (ATFP) by 2025 through application of Page 62 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) science for agriculture. CAADP provides the larger frame in which the S3A is operationalized. The S3A therefore is the broader framework for the implementation of the FAAP. S3A also provides African decision-makers with the rationale for increased investments in science and technology. The S3Ais about the necessary transformation of national STI institutions in order to achieve the desired social and economic transformation of Africa. The S3A is an organizing framework of issues, science options, and partnerships to bring about that desired future. It refers to the science, technology, extension, innovations; policy and social learning needed for Africa to meet its evolving agricultural development goals. The Agenda identifies the key strategic issues with an impact on science and agriculture and presents a suite of high-level actions for increasing the contributions of science to the development of agriculture in Africa at the local, national, regional and Pan-African levels. New topics warranted attention: Private-sector engagement. Increased global interest in public- and private-sector investment in agricultural research and development resulted in the G8 announced formation of the New Alliance for Food Security and Nutrition, at its Camp David Summit in May 2012. FARA was designated together with the CGIAR as a co-leading institution for the Alliance work- stream on technology innovation platforms. There was a need for FARA to deepen its efforts to engage the private sector in research, extension and education. Planning capacity. It was widely accepted that Africa’s poor agriculturalperformance is traceable to inappropriate policies and weak planning processes. There was a heightened need to catalyze the actions necessary to develop the capacity for Africa to conduct its own foresight and associated strategic analyses. Africa is home to the world’s youngest population. FARA had demonstrated its commitment to supporting Africa’s efforts in making the most of the demographic bonus of having a young population. FARA had hosted the Africa chapter of the Young Professionals in Agricultural Research for Development (YPARD). Gender Agenda. To ensure a proper commitment to addressing gender, FARA’s Gender Strategy (approved in 2013) was an integral component of FARA’s Strategic Plan and MTOP II. The Gender Strategy presents the broad strategic framework within which FARA was to pursue the goal of mainstreaming gender in facilitating the improvement of agricultural productivity in Africa. Malabo Declaration and financial challenges triggering more changes. In May 2014, the African Union Heads of State issued a Malabo Declaration that shifted the focus of the CAADP Pillar IV to country level development and gave greater focus on the S3A as the overarching framework for three CPs. FARA’s focus was to focus more on keeping strengthened ARD actors connected and integrated programmatically in order to avoid debilitating fragmentation of the African AIS. FARA’s financial situation became constrained. As of 2018, the EC remained as the only contributing donor partner to the MDTF for the period 2014-2018. The Netherlands also extended its participation in the MDTF for the same period but did not provide a new contribution. In 2014, FARA opted to implement the low case financial scenario. FARA were to re-assess its work plan and operational costs (including staffing) with a view towards downsizing to fit the available resources should no additional funding materialize by the second year of post-restructuring. Eventually, the resources did not materialize, and FARA’s ability to meet its key performance indicator targets and the development objective had become a major challenge. In 2015, FARA chose to revise the entire RF, including PDO, PDO and intermediate indicators and targets, in line with the revisions to the MTOP and revised Strategic Plan that was recently approved by its Board of Directors. FARA began to unofficially measure its progress against the revised RF to ensure it was able to report results against the Page 63 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) changes. Subsequently, effective in 2016, FARA had to critically reassess its strategy, operational plans and organizational structure. Following the recommendations of a 2015 institutional and functional analysis, the FARA Secretariat adopted a new organogram, in line with its available resources. On December 31, 2015, all FARA staff contracts were terminated and those staff that were offered and accepted the same or similar positions starting January 1, 2016, did so under the terms of a new salary and benefits package approved by FARA’s Board. The new structure includes two Directorates for Research and Innovation (DRI); and Corporate Partnerships and Communication (DCPC). DRI oversees FARA’s technical program and replaces three previous directorships, whereas DCPC deals with governance, partnerships, communication and resource mobilization. The activities center on three core areas: (i) Knowledge Management and Decision Support, (ii) Capacity Development for the Implementation of S3A and (iii) Innovation Systems and Partnerships. Source: Project Appraisal Document (2013), Mid-Term Review (2016), Restructuring Paper 2017 (Draft), Science Agenda for Agriculture in Africa. Page 64 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 9. ADDITIONAL INFORMATION ON ACHIEVEMENTS This Annex provides additional information on: (1) Achievements, providing a summary of the overall key results (not captured by the Results Framework) and Information on the ‘extra’ PDO and IR Indicators, and (2) Selected project highlights (as identified by the stakeholders). 1. Overall Key Achievements Overall key results in 2009-2013 (Table below with a summary). Overall, the selection of PDO outcome indicators did limited justice to FARA MDTF interventions. The RF does not capture many important achievements during the first four years. FARA gradually established itself as the continental IAR4D umbrella, representing the IAR4D actors at the continental and global level. FARA developed the CAADP Pillar IV strategy. By the end of 2013, FARA had completed the development of S3A that provided the overall framework for the science agenda and served as the basis of MTOP2 phase. FARA became e.g. the co-lead institution for G8 affiliated technology Innovation Platforms (IP). It also played a key role in the Dublin process that contributed to development of CAADP-CGIAR Alignment Initiative. In consultation with stakeholders, FARA catalyzed e.g. SSA CP which has become a successful research project integrating IAR4D approaches. Given the MDTF core funding, FARA was considered a trusted partner, contributing to many bilateral capacity-building and research programs (such as UniBRAIN, DONATA, RAILS, SABIMA, SCARDA, PAEPARD). However, the RF (excluding component 2) may not allow this kind of attribution to be made easily. FARA also played a substantial role (e.g. advocacy to gain traction among the right stakeholders, facilitation of stakeholder engagement, support for set-up of governance structures, initial financial support, and often resource mobilization) in building of an African wide network of AIS related actors. FARA supported the establishment of continental organizations, such as AFAAS (the African Forum for Agricultural Advisory Services), the Pan-African Farmers Forum (PAFFO), Pan-African Agribusiness and Agro-Industry Consortium (PanAAC), and Pan Africa NGO Consortium (PANGOC) along with SROs (existing ASARECA and CORAF and new NASRO and CCARDESA) - to lead different aspects of agricultural development37. One key achievement was inclusion of education actors for the first time within the AIS network working closely with RUFORUM and ANAFE to create TeamAfrica38, strengthening and bringing further attention to capacity-building. Overall key achievements in 2014-2018. FARA continued being visible both at the continental and global levels, most actors stating (institutional analysis 2017) that FARA had strengthened the voice of Africa in global platforms. While FARA began serving as the technical arm of AU and RECs, it still has room for improvement in influencing policy (64.5 % effectiveness) and capacity-building (55%) among its key constituency. Capacity-building partnerships continued, some being scaled-up by SROs39. FARA also continued streamlining the Secretariat and its core functions to better match the available resources and mandate. Appreciation for the FARA supported (with or without SROs) CAADP-FAAP integration into country NAIPs has grown, reaching more than 40 countries. Coordination and collective action are at the heart of strengthened AIS capacity. None of the indicators capture this. Previous and current studies/interviews indicate collaboration to date has generated better skills and appreciation for 37 Based on an institutional analysis (2017) AFAAS has still limited visibility among the IAR4D actors while it is known at the global level. Of the SROs, CORAF has the greatest capacity, visibility and policy level influence. CCARDESA and NASRO are still in the early stages. 38 However, TEAM Africa is often seen as duplicating a large part of RUFORUM and ANAFE. 39 The formulation of the WB financed regional WAAPP, EAPP, and APPSA serve as examples of capacity-building projects scaled up by the SROs and countries. The countries involved were initially engaged by FARA-led (bilaterally funded) projects such as PSTAD (DONATA & RAILS) and SCARDA. In the SADC region, the SADC secretariat extended the approach to EU-funded projects such as ICART and the French-funded collaborative research support project FIRCOP. The institutions involved continued interventions within the context of CCARDESA, CORAF and ASARECA, leading to the evolution of the IP work into the IAR4D. Page 65 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) benefits arising from sharing and collaboration (among NARIs), albeit slowly. SROs are also better at collaborating with their constituency than before. However, they also indicate that while FARA collaborated with SROs on technical aspects (e.g implementation of capacity-building projects, technology dissemination and IPs), it often worked in isolation and sometimes implemented activities directly with NARS. SROs on the other hand focused working with their constituency and did not engage in across region collaboration. Many reasons affected this working relationship, capacity being the main one. The majoity of FARA initiatives have focused on developing, testing or groung-truthing new approaches. In situations where SROs had only few pilot countries to choose from, the SROs occasionally recommended FARA to engage with the countries. During 2015-2017 in particular, low capacity and changes within SROs themselves affected implementation (e.g., the AHC-STAFF initiative) significantly. FARA led the consultative development of S3A as well as the Consortium that aimed at leading the follow-up implementation of S3A. Due to lack of traction among key partners (mostly SROs), the Consortium approach collapsed in 201740. There is still significant room for improvement in overall coordination and value-added activities. Towards the end, effort was also made toward developing a continent-wide standardized M&E system. While capacity varies significantly among SROs, they all noted improvements in their M&E capacity41. Summary of Key FARA Achievements (2009-2018) Achievement area MDTF 2009-201342 MTOP2 2014-2018 Leadership and ownership Visibility of African AIS at continental and global levels As on the left plus for AIS CAADP Pillar IV Strategy and S3A development Consortium on S3A attempted – did not receive support by key Dublin process and other global representation members (SROs)43 G20 Tropical Agricultural Platform (TAP) Technical arm of AU and RECs G8 technology innovation platforms with the CGIAR FARA is a go-to-place on AIS in Africa but faces competition Science Week from other continental organizations Reform of institutions and FAAP development (2006) and promotion and adoption of FAAP Promotion & adoption of FAAP by countries (40+) and SROs (3) services on AIS by countries (30), sometimes together with SROs Expansion of previous initiatives plus foresighting for decision- Promotion of Innovation to Impact Pathway concept, user-led making Improved capacity on AIS processes, IAR4D approaches e.g. in SSA CP, gender mainstreaming in IAR4D, policy and evidence for decisions Capacity needs assessments and support to NARS and later Capacity needs assessment (2006, 2012) and partnership with NAIS, organizational management trainings bilateral capacity-building and research programs (SCARDA, NARS receive most capacity-building from international RAILS, DONATA, SABIMA, UniBRAIN, PAEPARD) partners via TBAs Building of a network of AIS Establishment of FARA as a continental coordinator More streamlined and focused FARA Secretariat actors & establishment of Contributions to establishment of AFAAS, CARDESA, NASRO, Continued collaboration on innovation approaches. SWOT continental and regional PAFFO, PanAAC, PANGOC and support to other SROs analysis organizations Inclusion of education sector actors for innovation capacity Regional coherence Platforms for knowledge, technology, policy advice and capacity Platforms for knowledge, technology, policy advice and exchange capacity exchange & steps towards continental M&E Collaboration efforts led by FARA with many organizations, Collaboration efforts led by FARA with many organizations, however, mostly on technical matters. Overall coordination and however, coordination, collective action and trust still limited – collective action and trust still limited most SROs working with their own constituency 40 SROs were highly involved in the development and consultations of the S3A. Given the changes within the SRO leadership and technical staff, the institutional memory as well as ownership of the coalition have been partly lost. Some SROs treat S3A as a FARA led TBA rather than the main CAADP instrument for implementation of CAADP at the country level. Among countries, traction for S3A is better. 41 Efforts were made to harmonize FARA’s M&E with that of the SROs and AFAAS. This however was not fully achieved because the budget required to develop a continent-wide system was not approved. 42 Some of the activities have been bilaterally funded and FARA has played a role. 43 Support by IFAD and EC planned in 2018-2019. Page 66 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Increased investment to Greater emphasis on regional-continental ARD investments – The investments grew by 47% from 2000 to 2014 whereas ARD previously only about US$25 million (of the total US$ 1 billion) research intensity was reduced from 0.68% to 0.46% in 2014. 44 targeted at regional-continental activities 2. Intermediate Results Intermediate Indicators (2009-2013). The IR1 of ‘Strengthened institutions’ focused on advocacy, facilitation, training and support to countries to sign a country compact (alignment with FAAP) either directly or via the SROs. Those targets were met (see discussion associated with PDO1 outcome). However, attribution to FARA remains somewhat unclear, given SROs were also involved. The IR2 of ‘Improved access to knowledge and technology’ targets were fully met. Data indicates that many countries along with SROs and continental organizations exchanged and cross-linked their knowledge systems with each other. Data is not available on use of these knowledge systems, however. Similarly, the IR3 of ‘Decision-making tools for policy’ targets were mostly met. While data on actual use is not available, the institutional analysis (2017) pointed to 70% of national and sub-regional actors using these products. This also indicates that FARA’s communication strategy worked quite well during the MDTF. IR4 of ‘Improved capacity’ is at the center of FARA’s work. FARA carried out need assessments (2006, 2012) and select capacity-building activities (e.g. on organizational performance, Tropical Agriculture Platform initiative). Most of the capacity-building activities were bilaterally funded Time-Based Activities (TBA) that FARA promoted through the SROs, in alignment with CAADP-FAAP process or directly at the country level. Due to common time lag between capacity-building efforts and impact, the efforts had still limited effect on the development of agricultural innovations by targeted NARS45. However, advocacy and training on gender influenced targeted NARS. The project was very successful with IR5 of ‘Improved support for agricultural IPs in Africa’ and far exceeded all targets on key IAR4D approaches, namely IPs and strategic partnerships-alliances, as well as development and use of relevant guidelines. Sustained and exceeding country demand for IP approach reflects a change in awareness and capacity on AIS at the country level. Intermediate Indicators (2014-2018). FARA made reasonable progress towards meeting its IR targets (Dec 2017 values, Annex 1). The IR1 of ‘African agricultural stakeholders determining how the sector should be transformed and establishing the needed collective actions’ targets on IPs and other Strategic Alliances were nearly met. The demand for IPs by countries is steadily growing, indicating improved awareness, experience/capacity and hopefully sustainable scaling-out of AIS approaches on the ground. The superior outcomes from the SSA CP have driven this demand. The IP approach has been mainstreamed in ARD programs. Participation rates (70-100%) in AIS centered activities also reflect those found in previous surveys46. 44 ASTI data. Research intensity = agricultural research spending as a share of agricultural gross domestic product (AgGDP). The target for Africa is 1%. https://www.asti.cgiar.org/ 45 Strengthening Capacity for Agricultural Research & Development in Africa (SCARDA) program (2008-2010) aimed to improve the capacity and performance of participating 12 NARS in 10 countries in key areas of their AR4D functions. While most activities focused on degree programs, it also offered short courses on such topics as farmers’ participatory research, AIS approaches and IPM. Source: SCARDA End-of -Pilot Phase Report. FARA 2011. Page 67 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) The IR2 of ‘Improved capacity’ targets on awareness of select stakeholders’ capacity needs and improvements in select capacity areas were nearly met (for details, see discussion in pre-structuring). A total of four CoPs exist and traction remains good. The project also mostly met or exceeded its IR3 of ‘Enabling environment’ targets on ‘Improving stakeholders’ access to and use of relevant new knowledge for awarene ss-building and decision-making (e.g., foresighting). Despite good results, feedback also pointed to limited results with the knowledge management activities. RAILS and improvements in ICT connectivity were however acknowledged by the constituency. It is possible that the momentum created during the MDTF phase was not continued during the MTOP2. Improvements in ICT connectivity are a result of a strategy to increasingly rely on ICT for knowledge sharing and the emergence of new and readily accessible tools. 3. Selected FARA MDTF Highlights: FARA’s stakeholders high-lighted e.g. IPs, the Science Week, UniBRAIN and Innovation Platforms as successful initiatives. The triannual Africa Agricultural Science Week (AASW) and the FARA General Assembly meeting is the principal forum for all stakeholders in African agriculture STI to reflect on their achievements and craft strategies and actions aimed at enhancing the contribution of agriculture STI towards accelerating the continent’s economic and social transformation. The sixth AASW and FARA GA, was held in Accra, Ghana in 2013 and the last one in Kigali, Rwanda in 2016. Participation has been very high (also verified by studies) and considered useful. While the Science Agenda for Agriculture in Africa was a large initiative and a significant accomplishment, traction among stakeholders has varied. The Science Consortium 47, led by FARA, has not received sufficient support among the SROs. UniBRAIN was high-lighted as a successful Danida funded, FARA developed initiative. The Danida funded and FARA developed UNIBrain programme is a model testing new science-driven models at continental level. It aimed: 1) to support and commercialize agribusiness innovations; 2) to enhance university graduates’ entrepreneurial skills and employability; and 3) to share and upscale the innovative outputs, experiences and practices. The UniBRAIN introduced a business incubation model that relies on tripartite collaboration among universities, research organizations and private businesses to jointly operate co-owned business incubators. Six Agribusiness Innovation Incubator Consortia were launched in Ghana, Mali, Zambia, Uganda and Kenya facilitated by a team of partners: ANAFE, PanAAC, ABI-CRISAT, ASARECA, CCARDESA and CORAF/WECARD. By the end of 2014, UniBRAIN had facilitated the commercialization of 43 agribusiness technologies, supported 111 agribusiness SMEs, incubated 138 start-up entrepreneurs, created 1,869 jobs on aggregate terms and empowered 738 students (of whom 274 were female 733 were youth) in tertiary institutions through internships, industrial attachments and improved agribusiness curricula. Other achievements under UniBRAIN include the launch of the gender tool kit for agribusiness incubation; the incubators continued to recruit new incubates, broker new partnerships, re-appraised their business models, mobilized funding, and embraced new technological innovations. By the end of 2015, the UniBRAIN had developed a sustainability strategy to replicate the agribusiness model in other African countries. The program has evolved into an independent, self-supporting entity that will continue activities beyond 2015 through the African Agribusiness Incubation Network (AAIN). Innovation Platforms approaches. Over the years, FARA has piloted, tested and advocated for adoption of demand-driven approaches to addressing innovation. The achievements around Innovation and Agricultural Research for Development (IAR4D) have contributed significantly to the pursuit of increased productivity, value added and market access. One of the Consortium coordinated the S3A development. Follow-up work on Consortium aimed at transforming the continental apex into a Consortium (FARA, SROs, 47 The AFAAS) format. Page 68 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) key achievements are the Innovation Platform (IP) concept and scaling out in the member countries. FARA was designated, along with the CGIAR, as the co-lead institution for the work stream on technology IPs within the G8’s New Alliance for Food Security and Nutrition. Definition. While the operational definition of the IP varies, they are multi-stakeholder forums for information sharing and knowledge exchange along a commodity value chain with a view to enhancing agricultural productivity and socio- economic well-beings. They represent organizational models for stimulating innovation and development in agriculture and related sectors. FARA recognizes two main types of IPs, strategic and operational. Every country requires both types of IP. Through continent-wide initiatives, four IP approaches have been validated for intervening at different stages and ranges of the value chain.  1. A User-Led Process (ULP) which intervenes at the research conduct stage to bring in end-users to take the lead in defining the problem to be addressed by research.  2. The IP for Technology Adoption (IPTA) that engages actors within the technology generation and adoption continuum to ensure increased pace of technology adoption and impact.  3. The Integrated Agricultural Research for Development (IAR4D) where the IP engages all actors of the innovation system in platforms that ensure collective problem identification and development of solutions in the context of prevailing institutional bottlenecks to deliver economic gains; and  4. The Universities, Business and Research for Agricultural Innovation (UniBRAIN) model that integrates research, private sector and policy agribusiness, to provide commercialization of technologies and incubation of agribusinesses along specific value chains, job creation, improved livelihoods for increased impact. Uptake and impact. By mid-2016, over 290 active IP have been established reaching 5.4 million farmers across Africa. The income of individuals that participated on the IAR4D IP averaged US$3.73/day compared to the baseline income of US$1.61. Income of participating women increased by 326%; food security improved by 324%; wealth distribution by 5%; household income by 232%; income inequalities as measured by the Gini Coefficient decreased by 6%; and productive assets value increased (FARA MTR 2016). By the end of 2018, FARA had supported implementation of more than 400 IPs in several research projects and countries. Dissemination: Several products have been developed to guide the IP process. The Electronic Platform eRAILS captures and shares information from the IP to ensure farm-level capacity and productivity. Source: MTR 2016; eRAILS; https://faraafrica.org/wp-content/uploads/2017/12/Agricultural-Innovation-Platforms.pdf Page 69 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) ANNEX 10. MONITORING & EVALUATION – SUPPLEMENTAL INFORMATION Page 70 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) M&E Issue 2009-2012 2014-201548 2016-2018 Baseline collection Very poor-quality RF (incomplete, missing Baseline at approval TBD, during 2014 baseline data updated at the components 2 and 3), unaligned between output level. Use and adoption study was published and MTOP Baseline data or targets not defined at (informal) Reliability of data outcomes and outputs, vague and hard-to- evaluation work to provide data for PDO baselines. approval measure PDO, PDO and IR level indicators were Verification protocol of mixed nature, outcome and output level. While baseline exists, a need to clarify beneficiary targeting and #s, Reporting system structured in a quantifiable output While most indicators were quantifiable, they improve benchmarking and change levels of planned interventions. format. TBAs follow individual results frameworks (RFs), Reporting were rarely qualitative. In the absence of specific targets, the interventions follow a and each submits a detailed progress report every 3 budget-based implementation than a target-based operation - months with an activity-by-activity status report, Initial challenges with setting up a relevant set of yearly allocations of money for certain activities utilized according including progress on milestones and indicator targets. indicators (simple, harmonized across to the plan. organizations, with baselines and targets), Despite harmonized reporting, there is often no collection of baseline, and overall timely meaningful identification of or follow-up to issue areas. The M&E Strategy, Performance Monitoring Guidance Manuals, a reporting against targets by FARA and the Achievements are primarily at the interim output level. revised reporting format and style based on M&E principles and implementing partners criteria was instituted. PDO indicators with baseline and targets available at approval but mostly not collected. All data collection and reporting against revised indicators (starting in 2010) while revision official in June 2012. IR indicators within Indicators (revised) reported from ISR #4 Yes Yes ISR onwards. Corrective action on Addressing issues as listed above. Improved design (2014): revised measurable PDO and IR indicators, The M&E Unit in collaboration with other units of the M&E design Revised PDO outcome indicators in June 2012 clear(er) RF causal chain, baseline collection, reporting against Secretariat revised the PDO, indicators, baselines and Corrective action taken to address the significant activities, outputs, outcomes and milestones, and comparison to targets for the MTOP 2R results framework (2016 challenges with the overall M&E. A results-based work plans and budgets. Harmonized guidelines and formats for all onwards). M&E in February 2011 together with DPs. M&E actors at different geographic locations. Following the Workshop, FARA and the Bank However, in March 2017 FARA further changes closely engaged to collectively work on further proposed: second PDO, two additional PDO level A revised FARA M&E Strategy for 2014 – 2018. The accompanying developing FARA´s PMF. The primary purpose indicators, and 17 additional intermediate indicators. Performance Monitoring Guidance Manual published in 2014 was to clarify the content and ensure adequate Further streamlining in collaboration with WB, resulting reporting on existing indicators, and to ensure 48 Expectations in 2014-2015: FARA’s M&E system was centered on tracking and reporting performance against activity milestones and output, outcome and im pact indicators. The implementing agencies used harmonized guidelines and processes. The design had the following features: (i) a detailed PMP with indicators and targets specified in the results framework as part of an embedded performance monitoring system at continental, sub-regional and national levels, (ii) an implementation monitoring system that assessed the degree to which the implementation process complies with work plans and budgets. (iii) all the data and information generated by the implementing agencies/units following well-defined reporting formats, (iv) refined Key Performance Indicators (KPIs) in the results framework, so that implementers report on milestones on a regular basis. There are higher level indicators in the results framework to link FARA outputs to the anticipated higher-level benefits. (v) With several new indicators, a baseline survey within the first 6 months of effectiveness. Page 71 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) data collection and ownership of the indicators outlined several concrete actions FARA and partners must take to in one PDO, three PDO level indicators and a total of by FARA, resulting in a noticeable improvement maximize impact and improve the way of doing business. twelve intermediate indicators. in data collection and M&E. Attention to Due to recruitment challenges, a cost-sharing The quality of M&E on the program of work, activities and budget effective M&E Lack of an M&E specialist on staff for about 2 arrangement for M&E actions with SROs. continues to be satisfactory. M&E products useful to understand implementation years (2011-2013), use of outsourced M&E for progress, or lack thereof, in the implementation of the program and the time being. to make the necessary adjustments. The quality of M&E continues to be satisfactory. The Unit also supported bilateral donor-funded projects through quarterly program performance review and M&E reports useful tools for the Secretariat to make budgetary and programmatic decisions and provide data to improve FARA field supervision. The M&E reports are useful management tools, if used decision-making. consistently. They help to identify problems on time and Omissions: (i) timely submission of reports required further effort, and (ii) the causal chain between outputs and outcomes continued put in place corrective measures. was somewhat unclear. Sustainability of n/a n/a Towards coordinated M&E functions among S3A M&E institutions. The M&E Unit with initial contact with ASARECA, CCARDESA and AFAAS to provide support in: (i) harmonization of RFs based on the FARA-level indicators; (ii) identification of standard indicators shared or collected by all institutions; (iii) harmonization of data collection tools; and (iv) preparations for enhancing the use of online data collection approaches. Use of Data M&E challenges partly triggered restructuring Impact Evaluations - a strong push particularly from DPs to show Communication impact on the ground. FARA’s impact is linked with the work of Institutional analysis of IAR4D actors – 2017 many actors, and many factors that are outside FARA’s control. The Effect Mid-term review in 2012, including Joint Review RF with indicators on improvement in the # of farmers and Milestones contributed to management decisions (positive/negative) (EPMR and MTR) of FARA and FARA self- scientists/ practitioners/policy makers reached. on direction of assessment  restructuring Annual surveys and meetings implementation and FARA Independent Evaluation Report 2008 -2013 M&E, incl. reporting on expenditures and milestones, triggered In- subsequent in December 2014*. depth Fiduciary and Value for Money Reviews and a Procurement interventions Review in 2016 Annual stakeholder meetings A mid-term report (self-assessment) in 2016 – used by the Bank Evidence on Management decisions Team to undertake a MTR of the MDTF. achievements * Although the final draft was not shared with the mission, the findings recorded in the first draft is worrisome and needs management attention. It is reported that the schedule for the visit of the evaluation team in some countries conflicted with public holidays, and in one case, e lections, and therefore the evaluation team’s mission was not fully accomplished. Out of 4500 FARA stakeholders invited to participate in the survey, only 43 responded, (less than 1%), despite the tremendous effort made by the Secretariat to encourage participation. Page 72 of 74 The World Bank Forum for Agricultural Research in Africa (FARA) Trust Fund (P112684) Page 73 of 74