Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004886 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF0A1558 ON A SMALL GRANT IN THE AMOUNT OF USD 3 MILLION TO THE MINISTRY OF ECONOMY AND DEVELOPMENT PLANNING FOR Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) 31 May 2019 Energy & Extractives Global Practice Africa Region Regional Vice President: Hafez M. H. Ghanem Country Director: Soukeyna Kane Senior Global Practice Director: Riccardo Puliti Practice Manager: Christopher Gilbert Sheldon Task Team Leader(s): Silvana Tordo ICR Main Contributor: Silvana Tordo ABBREVIATIONS AND ACRONYMS MFB Ministry of Finance and Budget MEF Ministry of Environment and Fishery MPE Ministry of Petroleum and Energy SHT Chad National Oil Company PIU Project Implementation Unit TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 II. OUTCOME ...................................................................................................................... 9 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 14 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 15 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 15 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 17 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 25 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 26 ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Enhanced Capacity towards Sustainable Petroleum P155829 Sector Management Country Financing Instrument Chad Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Projet d’Appui à la Formation des Agents de Ministry of Economy and Development Planning l’Administration Publique au Tchad Project Development Objective (PDO) Original PDO

The objective of the project is to facilitate the efficient management of petroleum resources by strengthening the technical capacity of the beneficiaries.

Page 1 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) FINANCING FINANCE_T BL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-A1558 3,000,000 3,000,000 2,982,861 Total 3,000,000 3,000,000 2,982,861 Total Project Cost 3,000,000 3,000,000 2,982,861 KEY DATES Approval Effectiveness Original Closing Actual Closing 20-Nov-2015 11-Dec-2015 31-Dec-2018 31-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 20-Apr-2018 2.02 Change in Results Framework Change in Components and Cost KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 28-Jun-2016 Satisfactory Satisfactory 0.00 02 08-Dec-2017 Moderately Satisfactory Moderately Satisfactory 0.29 Page 2 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Paul Noumba Um Soukeyna Kane Senior Global Practice Director: Anita Marangoly George Riccardo Puliti Practice Manager: Paulo De Sa Christopher Gilbert Sheldon Task Team Leader(s): Ilhem Salamon Silvana Tordo ICR Contributing Author: Silvana Tordo Page 3 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context 1. Chad’s economy is heavily depended on petroleum revenue. Since the start of oil production in 2003 the relevance of the petroleum sector in Chad has steadily grown. The economy, previously agrarian, saw a per capita GDP increase from about $220.8 in 2002 to about $669.9 in 2017, well above other low-income countries (although still far from average Sub-Saharan Africa). However, the combined effect of the drop in international oil prices in 2014 and the weak security environment left the country in deep recession, resulting in sharp cuts in public expenditure, low foreign direct investment, and a loss of income caused by the disruption in cross-border trade with Nigeria. Growth dropped from 6.9 percent in 2014 to -6.3 percent in 2016 and -3.0 percent in 2017. In 2018, increasing oil prices and production, combined with higher agricultural output underpinned a gradual recovery. Oil GDP increased from -11.2 percent in 2016 to 10.9 percent in 2018 while non-oil GDP rose from -6.7 percent to 1.0 percent in this period. 2. Harnessing the petroleum sector for fiscal sustainability is a priority for Chad’s long-term development strategy. Over a decade of reliance on oil has somewhat reduced incentives to diversify the economy and promote competitiveness, leaving the country more vulnerable to exogenous shocks. However, paradoxically strengthening economic resilience, in the short to medium term, will inevitably be driven by the petroleum sector. Many factors affect Chad’s ability to improve the benefits derived from the exploitation of petroleum resources. Some of these factors are exogenous such as the oil price and the cost of equipment and material. Other are internal, such as efficiency gains, attractiveness to investors, and good governance. The petroleum sector could also be used to reduce Chad’s exposure to price and production- related shocks, as well as to lay the basis for a more diversified economy, which in turn would increase resilience to exogenous shocks, create productive jobs, and lay the foundation for sustainable growth. 3. Chad's upstream industry is relatively immature, with a mere 15-year production history and considerable exploration upsides. The country has a proven petroleum system, with large basinal areas for further exploration. According to industry experts, Chad may hold around 40 Bn Bbl of oil and 40 Tcf of gas of potential yet-to-find, conventional resource, which suggests that further discoveries should be possible. In addition, only 40 percent of the original proved reserves of 1.5 Bn Bbls of oil are said to have been produced from the production licenses in the Doba basin, and undeveloped, but discovered oil in place is estimated to be 475 MM Bbls of oil equivalent, of which about 130 MM Bbls oil equivalent could be considered as contingent resources. In this context, it is essential for Chad to develop and retain in house capacity to encourage further exploration and to sustainably manage its petroleum production and associated revenue. 4. While it is true that Chad can learn from the experiences of more established jurisdictions, capacity constraints and limited budgets often inhibit its ability to implement best practice. The Ministry of Petroleum and Energy (MPE) is tasked with the oversight of petroleum operations by providing policy direction and ensuring compliance by investors with legal, regulatory and contractual frameworks. The MPE represents the State in all activities governed by Petroleum Law (Law no. 06/PR/2007) and its implementing regulations. The MPE shares responsibilities with the Ministry of Finance and Budget (MFB) on taxation matters, and the Ministry of Environment and Fishery (MEF) on environment protection and conservation. The MPE is not yet staffed with experienced professionals who could proactively ensure that the legal and contractual provisions are met by operators and investors or could develop and informed option on the Page 4 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) nature and volume of reserves and associated fiscal revenue being negotiated in the context of a new production sharing contract. However, frequent institutional reforms and non-competitive public wages hinder the MPE’s ability to attract and retain experienced professional staff. 5. Technical assistance projects often focus on procuring highly skilled international experts to provide advisory services to the government and on-the-job training. Lessons learned from these projects suggest that while advisory services are generally useful to tackle urgent and complex sector activities, rarely the outcome of such support benefits local capacity enhancement efforts. Acknowledging this challenge, the MPE has begun to implement a comprehensive training program to support capacity building for its technical, environmental, economic and financial staff engaged in the oversight of upstream petroleum operations. At the time of project preparation, MPE’s exploration and production staff had been formally assessed by an internationally renowned training firm (NEXT, a branch of the Schlumberger Group), some had been selected to receive training in geoscience and reservoir engineering, as well as English language training which is the common language of the oil industry. However, further capacity enhancement was considered necessary, particularly on project and field management, legal and contractual compliance, economic and financial evaluation, as well as environmental management and the development of tools to mitigate the risk of conflicts associated with petroleum development. Project Development Objectives (PDOs) 6. To facilitate the efficient management of petroleum resources by strengthening the technical capacity of the beneficiaries. The PDO has remained unchanged throughout project implementation. Key Expected Outcomes and Outcome Indicators 7. The expected outcome of this Project was the improved technical capacity of the beneficiaries, as measured by two PDO level indicators and five sub-indicators measuring capacity enhancement in three core technical areas. These indicators and sub-indicators are detailed in Section II and summarized below: a. Total number of project beneficiaries trained, with a minimum target of 30 trainees, and the following sub-indicators: i. Staff trained in exploration-production; ii. Petroleum Ministry staff trained to monitor contractual and regulatory compliance of petroleum field development operations; iii. Number of female project beneficiaries trained; iv. Staff trained in economic and financial management; v. Staff trained in stakeholder engagement and conflict prevention. b. Percentage of project beneficiaries who feel training reflected their needs, with a minimum target of 70 percent. Page 5 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) 8. Until March 2018, the project included another PDO level indicator - Increased capacity for stakeholders' and citizens' engagement in the petroleum sector. This indicator was to be measured by: (i) the percentage of staff trained in stakeholders' engagement and conflict prevention; (ii) the publication and adoption of a guidance note on effective stakeholders' engagement; (iii) annual reporting on petroleum revenue; and (iv) the percentage of issues related to non-compliance with contractual obligations annually identified through tools developed in the project. However, on the Government’s request, the project was restructured. Capacity building for stakeholders’ and citizens’ engagement and indicators related thereto were dropped due to the inability to attract service providers in this area and a change in Government priorities. In fact, by 18th of August 2017, the Government’s tender resulted in only one company providing an offer. A second tender was launch, which was unsuccessful. In December 2018, the Government completed a self- assessment of its compliance with the Extractive Industry Transparency Initiative (EITI) – a critical governance and stakeholders’ engagement tool in Chad – and identified the lack of a functioning cadaster among the shortcomings. A functioning petroleum cadaster was considered a critical tool for building oversight capacity, as it ensures consistency and transparency in the allocation and management of petroleum exploration and production rights and the monitoring of contractual obligations. Sub-component 1.4 – “Capacity Building for Stakeholders’ Engagement and Conflict Prevention” – was dropped together with the indicators described above. A new activity – “Initial assessment for the establishment of a petroleum cadaster” – was adopted in stead, as measured by an intermediate indicator (scoping study and preliminary system architecture analysis). Components Component 1: Training and Capacity Development 9. This component included a competency audit to determine the knowledge and skills of the pool of beneficiaries compared to discipline-specific performance criteria, which purpose was: to inform the design of training curricula tailored to the capacity and needs of the participating institutions; and to ensure that the selection process would rest on objective parameters (a minimum level of relevant knowledge and the relative learning capacity of the candidates). Four sub-components were envisaged: a. Sub-component 1.1: Training in Exploration-Production, aimed at enhancing the participant’s technical capacity to supervise fields operations and to review reserves assessment to support contract negotiations. Each training module was designed to target skills gaps identified through the competency audit. Topics to be covered by the training course included: i. Geoscience and reservoir engineering covering geology, geophysics, petrophysics, as well as reservoir geology and engineering; ii. Drilling and wells management covering drilling techniques, equipment and procedures, completion engineering, fluid analysis, well interventions and pressure control techniques; iii. Fields operations, which include operation of production facilities, safety, health and environment, equipment, maintenance and inspection, as well as process and layout engineering; and Page 6 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) iv. Projects and logistics, covering development engineering, construction superintendence, contract negotiation, procurement and management among others. b. Subcomponent 1.2: Training in Economic and Financial Management, aimed at enhancing institutional capacity to forecast and manage oil revenue. Training was designed to address skills gaps identified through a competency audit, and covered the following subjects: i. Upstream economics, which covers upstream economics and management, exploration-production contracts and fiduciary issues, economic analysis of exploration-production projects; ii. Finance and management, including financial management, economic modelling of investments, oil accounting and financial audits. c. Subcomponent 1.3: Institutional Assessment and Enhanced Legal Support to Petroleum Development, aimed at strengthening the participant’s capacity to manage issues related to the non-compliance with contracts, and to foster the alignment of model contracts with international best practices. This subcomponent also envisaged the review of institutional mechanisms for contract monitoring, enforcement of compliance, and dispute resolution with the view to provide recommendations and the rapid assessment of the institutional and organizational structure of the MPE, its affiliated companies and relevant structures within other ministries such as the National Technical Committee for Monitoring and Control to ensure that the training activities offered through this Project and through the Ministry’s own training program would be pertinent and cost-effective. Training modules were designed to address skills gaps identified through the competency audit, and would cover the following topics: i. Negotiation of concession and production sharing contracts; and ii. Settlement of litigations and claims in the event of a breach of contracts. d. Subcomponent 1.4: Scoping Study for the Establishment of a Petroleum Cadaster and Contract Management System (GeoCad). This sub-component aimed at supporting the MPE’s efforts to improve transparency and the management of its operation through the digitalization of geological, geophysical and petrophysical data, as well as licensing and contractual information and the informatization of management procedures. This component was restructured in March 2018 (see Box 1). Activities included: i. The preparation of a status assessment, covering initial data/ information inventory and collection, institutional and governance features, workflow and information flow description and analysis, current mapping infrastructure and IT system capabilities in Chad, and an estimate of the requirements/amount of scanning and digitization of existing data and information; ii. Data and institutional procedures inventory, with recommendations on the basic concepts for the future GeoCad (officers and public users) based on the characteristics of current and expected petroleum activities in Chad; and iii. the preparation of the strategy and basic action plan to transfer the current information (paper based) to a new data structure compatible with a GIS methodology. Page 7 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) Box 1. Component 1.4 - Original design. Until March 2018, component 1.4 focused on Capacity Building for Stakeholder Engagement and Conflict Prevention, which was to finance a study on the vulnerability of the petroleum sector to conflicts in the region (Lake Chad, Boko Haram) with the objective to strengthen the Government’s capacity to engage with communities and other stakeholders to prevent conflict associated with petroleum development in Chad. The study would: (i) identify the drivers that could create conditions of fragility and conflict as a result of petroleum development and determine the capacity of relevant institutions to manage those drivers; (ii) identify the vulnerability of the petroleum sector to conflicts in central Africa, particularly the effects on production and new investments; (iii) build greater understanding and consensus among stakeholders in order to prevent conflict related petroleum sector development; and (iv) provide a stand-alone guidance note on effective citizens’ and stakeholders’ engagement in the development of petroleum and resource management. The study would also inform (a) the development of tools and capacity to engage effectively with community and stakeholders’ including conflict prevention strategies such as holding consultations as well as design and implementation of a communication strategy and outreach plan, (b) capacity building for the supervision of the implementation of social and environment risk mitigation assessments and action plans, (c) the development of a thorough understanding of equitable and transparent benefit-sharing processes and best practices in petroleum legal and regulatory framework for sustainable petroleum development, and (d) capacity building of relevant institutions to address grievances related to distribution of revenues from the petroleum sector. However, in March 2018, Sub-component 1.4 was restructured due to the change in Government priorities and the inability of the PIU to elicit market interest in two subsequent tenders. In place of the capacity building for stakeholders’ engagement and conflict prevention, a new activity – the preparation of phase one of the scoping study for the establishment of a petroleum cadaster and contract management system (GeoCad) – was retained. Component 2. Project Implementation Support. 10. This component was meant to finance the incremental costs associated with the implementation of the Project. This included: (i) partial payment for a procurement expert and an accountant currently already embedded in the PIU to help with the additional fiduciary workload; (ii) technical experts who will help the PIU during the evaluation process associated with the procurement of the main consulting contracts; and (iii) the consulting services needed to update the project implementation manual and to perform the grant audit. Page 8 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) II. OUTCOME Assessment of Achievement of Each Objective/Outcome 11. The PDO of this Project - to facilitate the efficient management of petroleum resources by strengthening the technical capacity of the beneficiaries – is deemed satisfactory as confirmed by the achievement of all PDO level indicators. 12. The target of all PDO level indicators of the Project, which remained unchanged throughout implementation, were successfully achieved. a. PDO Indicator 1: Total number of project beneficiaries trained (Data Type: number). Result: 31 Sub-Indicators: • Staff trained in exploration-production (Data Type: percentage) • Number of female project beneficiaries trained (Data Type: percentage) • Petroleum Ministry staff trained to monitor contractual and regulatory compliance of petroleum field development operations (Data Type: percentage) • Staff trained in economic and financial management (Data Type: percentage) • Staff trained in stakeholder engagement and conflict prevention (Data Type: percentage) Assessment: The baseline for this indicator was “0”. The project supported the training of 31 staff chosen from among a pool of candidates from the MPE, the MFB, the National Petroleum Company (SHT) and the MEF. The minimum target for this indicator was set at 30 trainees, as this represented the pool of candidates that had been identified as high potential from previous training programs supported by the MPE. The idea was to build on previous training investment to ensure that a group of highly qualified government officials would be in place. Given the limited financial resources of the Government, a choice was made to focus such resources on training government officials with high potential. The baseline for all sub-indicators was “0”. Instead of setting a specific target for these sub-indicators it was decided that a competency audit would identify the trainees that displayed a minimum standard of knowledge in each of the three subject matters to be financed by the Project and had the highest potential for improvement. The competency audit was part of the contract for training services and was carried out in November 2017 by the French Institute of Petroleum. A detailed evaluation of the sub-indicators is provided below: • Staff trained in exploration-production. Of the 37 government officials that were assessed through the competency audit, 15 were selected for training (representing 40.5 percent of the beneficiaries), one person withdrew, and 12 achieved graduation (representing 85.7 percent of the participants). Since the project objective was to improve the technical capacity of the beneficiaries, the Page 9 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) achievement of this sub-indicator is to be assessed through the improvement in trainees’ knowledge, measured by comparing the average entry score (7.9 points) with the average graduation score (13.9 points), which suggests that overall the training was effective at improving the capacity of the participants, and by extension that of the Administration. • Number of female project beneficiaries trained. Overall 9 women qualified through competency audit and 3 were selected for training (representing 33.3 percent of the pool of candidates), one for each discipline. The paucity of women candidates reflects the traditional scarcity of women in the oil and gas sector, which is often more acute in developing economies. Since the project objective was to improve the technical capacity of the beneficiaries, the achievement of this sub-indicator is to be assessed through the improvement in trainees’ knowledge, measured by comparing the average entry score (8.5 points) with the average graduation score (13.7 points), representing a 61 percent improvement in core knowledge. • Petroleum Ministry staff trained to monitor contractual and regulatory compliance of petroleum field development operations. Of the 13 government officials that were assessed through the competency audit, 5 were selected for training (representing 38.5 percent of the beneficiaries), and 5 achieved graduation (that is, 100 percent of participants). Since the project objective was to improve the technical capacity of the beneficiaries, the achievement of this sub-indicator is to be assessed through the improvement in trainees’ knowledge, measured by comparing the average entry score (10.9 points) with the average graduation score (14.1), which suggests that overall the training was effective at improving the capacity of the participants, and by extension that of the Administration. • Staff trained in economic and financial management. Of the 27 government officials that were assessed through the competency audit, 12 were selected for training and 10 achieved graduation (representing 83.3 percent of the participants). This group represent 44.4 percent (12/27) of the pool of candidates that whose work require knowledge of economic and financial management. Since the project objective was to improve the technical capacity of the beneficiaries, the achievement of this sub-indicator is to be assessed through the improvement in trainees’ knowledge, measured by comparing the average entry score (9.7 points) with the average graduation score (12.9), which suggests that overall the training was effective at improving the capacity of the participants, and by extension that of the Administration. • Staff trained in stakeholder engagement and conflict prevention. Of the 13 government officials that were assessed through the competency audit, 5 were selected for training (representing 38.5 percent of the beneficiaries), and 5 achieved graduation (that is, 100 percent of participants). Since the project objective was to improve the technical capacity of the beneficiaries, the achievement of this sub- indicator is to be assessed through the improvement in trainees’ knowledge, measured by comparing the average entry score (10.9 points) with the average graduation score (14.1), which suggests that overall the training was effective at improving the capacity of the participants, and by extension that of the Administration. b. PDO Indicator 2: Percentage of project beneficiaries who feel training reflected their needs (Data Type: percentage) Result: 85 percent. Page 10 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) Assessment: According to the training completion report, trainees concurred on the adequacy of the curriculum and expressed satisfaction with the training. An internal assessment was carried out by the coordinator of project implementation unit (PIU), which found that 85 percent of the trainees reported satisfied with their training. The result is considerable higher than the target of 70 percent established at project design. The main difficulties expressed by trainees related to the fast pace and density of the training course, and the language of the training material (English as requested by the MPE) which required additional learning efforts from the trainees. c. Intermediate Indicator 1: Direct Beneficiaries of the Training Program have been selected through the Competencies Audit (Training in Exploration-Production) (Data Type: percentage) Assessment: This indicator was fully achieved (100 percent versus a target of 100 percent). All training participants from the MPE were selected following a competency audit carried out by the French Petroleum Institute. Trainees from all participating ministries were administered entry and graduation tests. d. Intermediate Indicator 2: Percentage of trainees who successfully completed training programs (Training in Exploration-Production). (Data Type: percentage) Assessment: This indicator was partially achieved (78.6 percent versus a target of 80 percent). Of the 15 trainees in exploration and production, 1 withdrew during training and two did not graduate. The trainees that did graduate showed considerable improvement, from an average entry score of 8.1 points to a graduation average of 15.7 points (on a scale of 1- 20). The training completion report highlights the wide heterogeneity in learning capacity and entry levels across participants. e. Intermediate Indicator 3: Direct Beneficiaries of the Training Program have been selected through the Competencies Audit (Training in Economic and Financial Management) (Data Type: percentage) Assessment: This indicator was fully achieved (100 percent versus a target of 100 percent). All training participants from the MPE were selected following a competency audit carried out by the French Petroleum Institute. Trainees from all participating ministries were administered entry and graduation tests. f. Intermediate Indicator 4: Percentage of trainees who successfully completed training programs (Training in Economic and Financial Management). (Data Type: percentage) Assessment: This indicator was fully achieved (83.3 percent versus a target of 80 percent). Of the 12 trainees that successfully qualified for the program, 2 were not able to graduate. The trainees group included 2 staff of the MFB and 1 staff from the national oil company (SHT). The 10 trainees that did graduate showed notable improvement, from an average entry score of 9.8 points to a graduation average of 13.5 points Page 11 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) (on a scale of 1- 20). The training completion report highlights the wide heterogeneity in learning capacity and entry levels across participants. g. Intermediate Indicator 5: Direct Beneficiaries of the Training Program have been selected through the Competencies Audit (Institutional Assessment and Enhanced Legal Support to Petroleum Development) (Data Type: percentage) Assessment: This indicator was fully achieved (100 percent versus a target of 100 percent). All training participants from the MPE were selected following a competency audit carried out by the French Petroleum Institute. Trainees from all participating ministries were administered entry and graduation tests. h. Intermediate Indicator 6: Percentage of trainees who successfully completed training programs (Institutional Assessment and Enhanced Legal Support to Petroleum Development). (Data Type: percentage) Assessment: This indicator was fully achieved (100 percent versus a target of 80 percent). All trainees that successfully qualified for the program were able to graduate. The trainees group included 2 staff of the MEF. The trainees that did graduate showed notable improvement, from an average entry score of 10.9 points to a graduation average of 14.1 points (on a scale of 1- 20). The training completion report highlights the wide heterogeneity in learning capacity and entry levels across participants. i. Intermediate Indicator 7: Rapid Assessment of institutional and organizational structure of MPME (Data Type: number) Assessment: This indicator was fully achieved (1 versus a target of 1). A rapid assessment of the institutional and organizational structure of the MPE, its affiliates and the relevant units within the Ministry of Environment and the MFB was carried out by the French Petroleum Institute to inform the design of the training curriculum. The assessment raised concerns over the principles that underpinned the design of the organizational structure (relevance to the task and coherence with available skills) and its implication on intra and inter-ministerial coordination. The lack of job descriptions was also noted. Overall the study recommended the need to review the institutional arrangements and proposed a training curriculum based on the functions that are usually assigned to staff in similar structures, considering the baseline skills emerged from the competency audit. j. Intermediate Indicator 8: Report on the initial assessment for the establishment of a petroleum cadaster completed. (Data Type: number) Assessment: This indicator was fully achieved (1 versus a target of 1). The report on the establishment of the petroleum cadaster is the first phase of a pre-feasibility study aimed at assessing the size, structure, and functionality Page 12 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) of a future petroleum cadaster and contract management system. The report focused on the status assessment (type, size, and flow of data) and strategy for migration of data to a GIS system. Overall Outcome Rating 13. The overall outcome rating is Satisfactory, based on: a. the efficient use of project resources: only 4 percent of expenditure was used for project administration versus a forecast of 6 percent; b. the percentage of graduation: 87 percent of participants achieved graduation versus a target of 80 percent; c. the improvement in knowledge: participants showed a sizeable improvement (44 percent) in participants’ technical knowledge scores from the average entry level of 9.4 points to the average graduation score of 13.5 points (on a scale from 1 to 20, with minimum pass score of 12 points; d. the percentage of participants that reported satisfied with the project was 85 percent, considerably higher than the target of 70 percent; and e. the acceleration in project implementation from March 2018. Other Outcomes and Impacts 14. Training is defined as “a planned process that is used to change attitude, knowledge, skill or behavior through the learning experience to achieve effective performance in an activity or range of activities. Its purpose, in the work situation, is to develop the abilities of individuals and to satisfy the current and future needs of the organization.” 1 Therefore the impact of this project on the overall capacity of the participating institutions can only be observed over time, as the benefit of the newly acquired knowledge are applied in practice, and the trainees’ exposure to new information and tools is passed down to the organization. The review of literature related to training strategies suggests that training programs and organization strategies are tightly interlinked. Training programs and strategies must be tailored to the specific organization, its mission, its culture, its priorities, its needs, its allocation of tasks and responsibilities, knowledge gas and vision going forward. To maximize the impact of a training program, a plan should be devised that links the newly acquired knowledge and skills to day to day activities of the trainees, and constantly reinforces the value of acquired knowledge through responsibility and peer learning. Institutional reforms and the design of incentive and continuous professional learning programs was beyond the scope of this project. However, the rapid assessment of organizational structure of the MPE, its affiliates and the relevant units within the Ministry of Environment and the MFB financed by this project informed the restructuring of the MPE in 2018, that resulted in the realignment and redefinition of roles as set out in Decree 429/PR/PM/MPE/2018. 1 Beardwell, I. and L. Holden, 2001. Human Resources Management. A Contemporary Approach. Essex: Pearson Education Limited. Page 13 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME 15. Despite a simple and straightforward project design, lack of capacity and cumbersome public procurement processes, particularly for large consulting contracts, contributed to slow the pace of project implementation. The project became effective in December 2015, but implementation activities effectively started in September 2017 when the contract for training services with the French Petroleum Institute received the required Government approvals. Table 1 provides a summary of the administrative steps associated with the procurement process. Another contributing factor was the relative inexperience of the PIU that, although established before this Project, was understaffed and had to be strengthened by recruiting an accountant and a procurement specialist who were not knowledgeable in World Bank procedures and systems. Furthermore, the PIU’s failure to utilize the World Bank’s STEP - Systematic Tracking of Exchanging in Procurement – notwithstanding training received contributed to approvals delays. 2 16. The restructuring of the project in March 2018 resulted in further simplification in project design and its realignment with the MPE’s policy priorities. Training activities started on March 12 and were completed on June 6, 2018. On the other hand, while the terms of reference for the scoping study on the petroleum cadaster finalized and agreed with the World Bank July 2018, it was not until October 2018 that a contract for services was awarded to a consultant, further underscoring the challenges that STEP poses for capacity constrained project implementation units. To tackle this obstacle, the Bank provided on-the-job training and access to STEP from the Bank’s office where reliable internet connection could be secured. In addition, the Government strengthened the PIU by allocating expertise from a more experienced project implementation unit anchored within the Ministry of Education. Table 1 – Administrative steps associated with the approval of the contract for training services Entity Role Date Public Procurement Service Opinion on the proposed award 21 July 2017 of the contract for training services to the French Petroleum Institute World Bank Non-objection 25 July 2017 French Petroleum Institute Contact signature 7 August 2017 Ministry of Economy and Contract is initialed 26 August 2017 Development Planning Ministry of Petroleum and Contract is initialed 4 September 2017 Energy Prime Minister and President of Contract approval 28 September 2017 the Republic of Chad 2 STEP was updated retroactively before project closing in November 2018. Page 14 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME 17. Bank performance in managing the project is rated ‘Satisfactory’. Initial risks were posed to the development outcomes due to delays in the Government procurement process and failure to comply with the requirements of STEP. Furthermore, the supervision of a small stand-alone grant in a sector where no engagement had been in place for 7 years also presented challenges. The Project was intended as a first attempt to re-engage in policy dialogue in the sector, and as such it required intense supervision beyond the resources normally available to small projects of this nature. However, these challenges were effectively addressed by the World Bank team and the recipient as the project was being implemented, as reflected by the acceleration in project implementation after since March 2018, and the disbursement ratio of 99.43 percent. 18. On the other hand, the lack of agreed targets for the PDO level sub-indicators in the monitoring and results framework makes it difficult to assess whether the project has achieved its full potential. This design gap was mitigated by the achievement of the trainees, and their assessment of usefulness of the training received (85 percent of trainees reported satisfied with the training). V. LESSONS LEARNED AND RECOMMENDATIONS 19. The following lessons were learnt through the project: a. Training plans should be aligned with specific mission goals of an institution. To determine the concrete benefits of training, clear connections between training and an institution’s goals need to be established. In other words, it is necessary to identify the mission and objectives related to performance. The institutional review of the MPE (the principal beneficiary of the Project activities) carried out by the French Institute of Petroleum in 2017 identified a lack of coherence between the structure of the Ministry (later reorganized), and the capacity of its employees to fulfil their roles. The lack of detailed terms of reference for each position was also identified as contributing factor to the overlap of responsibility across different job posts, and the difficulty in assessing skills gaps. A training program was therefore designed with reference to the skills and knowledge that staff in similar positions would need to display, and the reorganization of the Ministry was recommended. This was a practical approach aimed at developing a relatively broad set of competence which could be flexible enough to adapt to organizational changes over time and provided an immediate response to an urgent need perceived by the beneficiary. Next Steps: The importance of skills development and retention is recognized in decree 429/PR/PM/MPE/2018 on the organization of the MPE, which includes among the tasks of the Human Resources Department the identification of staff incentives, performance monitoring, and solutions for continuous professional development. Future training programs should be informed by an Institutional Capacity Assessment (or similar training needs analysis) of the MPE and of other entities that intervene in the governance and oversight of the petroleum sector. Such assessment should be carried out before a tender package is prepared (not as part of a contract for training services) and should provide clear design parameters (training strategy and its integration with the institution Page 15 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) human resources management system) and a baseline to measure training effectiveness. b. Training targets should be defined at the outset. Existing literature shows that, in addition to enhancing knowledge and skills, measuring training effectiveness is an important tool to boost employee engagement and retention. Results and measurements of past training also act as critical indicators while planning future training. Post-training quizzes, one-to-one discussions, employee surveys, participant case studies, and official certification exams are some ways to measure training effectiveness. The more data is collected on measurable outcomes, the easier it will be to quantify the return on investment for training activities. Metrics and training targets should be identified before starting the training. Various aspects of training can be measured: i) reaction indicators measure how learners have reacted to the training, the relevance and usefulness of the training; ii) learning indicators measure the knowledge and skills gained by trainees as a result of the training; iii) behavior indicators measure how the training has impacted the learner’s performance and attitude at work; and iv) results indicators measure the tangible results of the training such as reduced cost, improved quality, increased productivity, and employee retention. Reaction and learning indicators were selected to measure the effectiveness of this Project since its objective was to “strengthen the technical capacity of the beneficiaries”. The choice was appropriate given the intentionally focused nature of the training program (which was intended to build on a recently completed training program financed by the MPE) and its short duration. Next Steps: Future training programs should further consolidate the investment on improving the knowledge and skills of the “core group” of trainees while identifying institutional mechanism for the sharing of such knowledge and skills. This would allow the benefit of training to percolate through the organization in a cost-effective manner. In this case, the use of behavior and results indicators should be considered and linked to the Ministry’s human resources management plan. . Page 16 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Facilitate the efficient management of petroleum resources by strengthening the technical capacity of the beneficiaries Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Total number of project Number 0.00 30.00 31.00 beneficiaries trained 18-Nov-2015 18-Nov-2015 06-Jul-2018 Staff trained in exploration- Percentage 0.00 0.00 40.50 production 18-Nov-2015 18-Nov-2015 06-Jul-2018 Number of female project Percentage 0.00 0.00 33.30 beneficiaries trained 18-Nov-2015 18-Nov-2015 06-Jul-2018 Petroleum Ministry staff Percentage 0.00 0.00 23.30 trained to monitor contractual and regulatory compliance of 18-Nov-2015 18-Nov-2015 06-Jul-2018 petroleum field development operations Page 17 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) Staff trained in economic and Percentage 0.00 0.00 33.30 financial management 18-Nov-2015 18-Nov-2015 06-Jul-2018 Staff trained in stakeholder Percentage 0.00 0.00 0.00 23.30 engagement and conflict prevention 18-Nov-2015 18-Nov-2015 06-Jul-2018 Comments (achievements against targets): The total of trainees that undergo training under the project was 31, including 2 officials from the Ministry of Environment and Water, 2 officials from the Ministry of Finance and Budget, and one staff of the national oil company (SHT). A total of 26 participants successfully graduated from training. Candidate trainees from the Ministry of Petroleum and Energy were selected on the basis of a competency test administered to the staff of the relevant directorates (a total of 77 staff). Trainees from all ministries took an entry test and a graduation test. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of project Percentage 0.00 70.00 85.00 beneficiaries who feel training reflected their needs 18-Nov-2015 18-Nov-2015 06-Jul-2018 (disaggregated by training programs, gender). Comments (achievements against targets): According to the training completion report, trainees concurred on the adequacy of the curriculum and expressed satisfaction with the training. An internal assessment was carried out by the coordinator of PROFAB, and reported that 85 percent of the trainees reported satisfied with their training. The main difficulties expressed by trainees related to the fast pace and density of the training course, and the language of the training material (English as requested by the Ministry of Petroleum and Energy). Page 18 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) A.2 Intermediate Results Indicators Component: Component 1 - Training and Capacity Development Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Direct Beneficiaries of the Percentage 0.00 100.00 100.00 Training Program have been selected through the 18-Nov-2015 18-Nov-2015 23-Jan-2018 Competencies Audit (Training in Exploration-Production) Comments (achievements against targets): All training participants from the Ministry of Petroleum and Energy were selected following a competency audit carried out by the French Petroleum Institute. Trainees from all participating ministries were administered entry and graduation tests. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of trainees who Percentage 0.00 80.00 78.60 successfully completed training programs (Training in 18-Nov-2015 18-Nov-2015 06-Jul-2018 Exploration-Production ). Comments (achievements against targets): Of the 15 trainees in exploration and production, 1 withdrew during training and two did not graduate. The trainees that did graduate showed considerable improvement, from an average entry score of 8.1 points to a graduation average of 15.7 points (on a scale of 1- 20). The training completion report highlights the wide heterogeneity in learning capacity and entry levels across participants. Page 19 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Direct Beneficiaries of the Percentage 0.00 100.00 100.00 Training Program have been selected through the 18-Nov-2015 18-Nov-2015 23-Jan-2018 Competencies Audit (Training in Economic and Financial Management) Comments (achievements against targets): All training participants from the Ministry of Petroleum and Energy were selected following a competency audit carried out by the French Petroleum Institute. Trainees from all participating ministries were administered entry and graduation tests. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of trainees who Percentage 0.00 80.00 83.33 completed training programs (Training in Economic and 18-Nov-2015 18-Nov-2015 06-Jul-2018 Financial Management). Comments (achievements against targets): Of the 12 trainees that successfully qualified for the program, 2 were not able to graduate. The trainees group included 2 staff of the Ministry of Finance and Budget and 1 staff from the national oil company (SHT). The 10 trainees that did graduate showed notable improvement, from an average entry score of 9.8 points to a graduation average of 13.5 points (on a scale of 1- 20). The training completion report highlights the wide heterogeneity in learning capacity and entry levels across participants. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Page 20 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) Direct Beneficiaries of the Percentage 0.00 100.00 100.00 Training Program have been selected through the 18-Nov-2015 18-Nov-2015 23-Jan-2018 Competencies Audit (Institutional Assessment and Enhanced Legal Support to Petroleum Development) Comments (achievements against targets): All training participants from the Ministry of Petroleum and Energy were selected following a competency audit carried out by the French Petroleum Institute. Trainees from all participating ministries were administered entry and graduation tests. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of trainees who Percentage 0.00 80.00 100.00 completed training programs (Institutional Assessment and 18-Nov-2015 18-Nov-2015 06-Jul-2018 Enhanced Legal Support to Petroleum Development). Comments (achievements against targets): All trainees that successfully qualified for the program were able to graduate. The trainees group included 2 staff of the Ministry of Environment and Water. The trainees that did graduate showed notable improvement, from an average entry score of 10.9 points to a graduation average of 14.1 points (on a scale of 1- 20). The training completion report highlights the wide heterogeneity in learning capacity and entry levels across participants. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Page 21 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) Rapid Assessment of Number 0.00 1.00 1.00 institutional and organizational structure of MPME (number) 18-Nov-2015 18-Nov-2015 09-Nov-2017 Comments (achievements against targets): A rapid assessment of the institutional and organizational structure of the Ministry of Petroleum and Energy, its affiliates and the relevant units within the Ministry of Environment and the Ministry of Finance and Budget was carried out by the French Petroleum Institute to inform the design of the training curriculum. The assessment raised concerns over the principles that underpinned the design of the organizational structure (relevance to the task and coherence with available skills) and its implication on intra and inter-ministerial coordination. The lack of job descriptions was also noted. Overall the study recommended the need to review the institutional arrangements and proposed a training curriculum based on the functions that are usually assigned to staff in similar structures, taking into account the baseline skills emerged from the competency audit. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Report on the initial Number 0.00 0.00 1.00 1.00 assessment for the establishment of a petroleum 07-Mar-2018 18-Nov-2015 07-May-2018 31-Dec-2018 cadaster completed (number) Comments (achievements against targets): The report on the establishment of the petroleum cadaster is the first phase of a pre-feasibility study aimed at assessing the size, structure, and functionality of a future petroleum cadaster and contract management system. The report focused on the status assessment (type, size, and flow of data) and strategy for migration of data to a GIS system. Page 22 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) A. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1 Outcome Indicators Total number of project beneficiaries trained. 1. Direct Beneficiaries of the Training Program have been selected through the Competencies Audit (Training in Exploration- Production) 2. Percentage of trainees who successfully completed training programs (Training in Exploration-Production 3. Direct Beneficiaries of the Training Program have been selected through the Competencies Audit (Training in Economic and Financial Management) 4. Percentage of trainees who completed training programs (Training in Economic and Financial Management). Intermediate Results Indicators 5. Direct Beneficiaries of the Training Program have been selected through the Competencies Audit (Institutional Assessment and Enhanced Legal Support to Petroleum Development) 6. Percentage of trainees who completed training programs (Institutional Assessment and Enhanced Legal Support to Petroleum Development). 7. Rapid Assessment of institutional and organizational structure of MPE 8. Report on the initial assessment for the establishment of a petroleum cadaster completed Page 23 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) 1. All project beneficiaries were selected on the basis of a competency audit; 2. 87 percent of trainees successfully completed their training Key Outputs by Component 3. Trainees saw an average increase in knowledge of 44 percent (linked to the achievement of the Objective/Outcome 1) 4. The rapid assessment of the institutional and organizational structure of the MPE identified gaps that informed the reform of the MPE Objective/Outcome 2 Percentage of project beneficiaries who feel training reflected their Outcome Indicators needs Intermediate Results Indicators N/A Key Outputs by Component 1. 85 percent of project beneficiaries reported satisfied with the (linked to the achievement of the Objective/Outcome 2) training received. Page 24 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) . ANNEX 2. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Components Percentage of Approval (US$M) Closing (US$M) Training and Capacity 2.80 2.86 1.02 Development Project Implementation 0.20 0.12 59.6 Support Total 3.00 2.98 99.43 Page 25 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS In April 2018 the PIU recruited an independent consultant to assess the pertinence and results of the Project. Below is the translation in English of the conclusion section of the completion report. “The capacity building project for Enhanced Capacity towards Sustainable Petroleum Sector Management has the merit of having been initiated in a context that largely justifies its relevance. Its implementation – which has not encountered any insurmountable difficulties, due to the supervision by the World Bank and the experience of PROFAP – has also been satisfactory; both in terms of performance of activities and budget execution and efficiency. Moreover, although the assessment of effects and impact has proved somewhat problematic (mainly because of the lack of specific target for some project indicators, which has introduced a certain level of subjectivity in the assessment), the available information suggest that the Project has substantially contributed to the technical capacity of the Ministry of Petroleum and Energy, the Ministry of Finance and the Budget and the MEF. The Project also contributed to the process of establishing a geo-referenced oil cadaster in Chad. However, the expected impact of the Project is such a vast area cannot be achieved by this Project alone. The knowledge gained through the training sessions funded by the Project must be shared, since only about 30 people have benefited from it. This knowledge will not, moreover, be used for the expected purpose if the beneficiaries are not actively involved in the management of the petroleum sector. In addition, the answers provided by the project are partial. The training offered does not cover all the needs in this field, even within the targeted departments. Furthermore, the framing study is only a step in the process of setting up a geo-referenced oil cadaster. Finally, the economic context that motivated the implementation of the project is still very topical. The Chadian economy is still very dependent on oil and the unfavorable economic situation in the oil market persists, resulting in growing financial difficulties. In short, the Project retains all its timeliness and relevance. This initiative must therefore be pursued. And to enable it to have a more significant impact, the possible continuation that will be given to it must consider the main recommendations below formulated: 1. Equip future projects or programs with a comprehensive programmatic framework. The aim is not only to define in a coherent manner the objectives, the activities and the budget, as well as the results, but also to have a precise timetable and, above all, a comprehensive monitoring and evaluation system. The lack of such a monitoring-evaluation system is damaging in many respects. A program or project that does not have the main dashboards that are the results measurement framework – with a comprehensive plan and periodic monitoring-evaluation plans, and periodic indicators monitoring tables – and an implementation monitoring framework, has more chances of being implemented erratically. 2. Equip the project implementation unit with substantial and stable human resources projects throughout the implementation period. The project implementation unit must be able to bring Page 26 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) together and make use of all the necessary expertise (Coordinator, administrative and financial manager, accountant, procurement officer, monitoring-evaluation officer, support staff). This will ensure greater efficiency, with a sustainable workload, regular and frequent consultations, individual and collective activity reports, as well as updated data at regular (and short) time intervals. 3. List the actors who took part in the evaluated project (and especially those who have benefited from training) to integrate them actively in the management of the petroleum sector and to create conditions conducive to an optimal sharing of information (for higher visibility) and knowledge. This way any future training program will be able to capitalize on the achievement of this project and will be implemented more effectively. 4. Intensify advocacy with public authorities and consultations with key stakeholders to generate greater interest in similar programs.” Page 27 of 28 The World Bank Enhanced Capacity towards Sustainable Petroleum Sector Management (P155829) ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) Page 28 of 28