Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004992 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF0A3614 ON A SMALL GRANT IN THE AMOUNT OF USD 1.5 MILLION TO THE REPUBLIC OF CHAD FOR Data Production and Capacity Building in Chad (P159189) January 8th , 2020 Poverty And Equity Global Practice Africa Region Regional Vice President: Hafez M. H. Ghanem Country Director: Soukeyna Kane Regional Director: Elisabeth Huybens Practice Manager: Andrew L. Dabalen Task Team Leader(s): Aboudrahyme Savadogo, Aly Sanoh ICR Main Contributor: Aboudrahyme Savadogo, Aly Sanoh ABBREVIATIONS AND ACRONYMS AFDB African Development Bank CAPI Computer Assisted Personal Interviewing CPF Country Partnership Framework ECOSIT Enquête sur la Consommation des Ménages et le Secteur Informel au Tchad (Household Consumption and Informal Sector Survey in Chad) FCFA Franc de la Communauté Financière d’Afrique (Franc of the Financial Community of Africa) IDA International Development Association INSEED Institut National de la Statistique, des Etudes Economiques et Démographiques (National Institute of Statistics, Economics and Demographics Studies) MICS Multiple Indicator Cluster Survey MS Moderately Satisfactory NSDS National Strategy for the Development of Statistics NSO National Statistics Office PDO Project Development Objective TFSCB Trust fund for Statistical Capacity Building UNICEF United Nations Children's Fund US United States USAID United States Agency for International Development WAEMU West African Economic and Monetary Union TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 II. OUTCOME ...................................................................................................................... 6 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 12 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 13 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 14 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 15 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 20 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 21 The World Bank Data Production and Capacity Building in Chad (P159189) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P159189 Data Production and Capacity Building in Chad Country Financing Instrument Chad Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Ministry of Planning, Economy and International INSEED Cooperation Project Development Objective (PDO) Original PDO To strengthen the capacity of the national statistics organization to conduct living standards surveys that meet international quality standards and to publicly release resulting anonymized microdata. Page 1 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-A3614 1,500,000 1,500,000 1,500,000 Total 1,500,000 1,500,000 1,500,000 Total Project Cost 1,500,000 1,500,000 1,500,000 KEY DATES Approval Effectiveness Original Closing Actual Closing 05-Oct-2016 15-Feb-2017 30-Jun-2018 30-Jun-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 09-Mar-2018 0.63 Change in Loan Closing Date(s) Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Moderately Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 23-Feb-2018 Satisfactory Moderately Satisfactory 0.52 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Page 2 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) Country Director: Paul Noumba Um Soukeyna Kane Director: Carolina Sanchez Elisabeth Huybens Practice Manager: Andrew L. Dabalen Andrew L. Dabalen Aboudrahyme Savadogo, Aly Task Team Leader(s): Kristen Himelein Kastelic Sanoh ICR Contributing Author: Aboudrahyme Savadogo Page 3 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context 1. The Institut National de la Statistique, des Etudes Economiques et Démographiques (INSEED) was founded in September 2000 and is the national statistics bureau for the Republic of Chad . At appraisal, INSEED has collected several national household level surveys, including the “Enquête sur la Consommation des Ménages et le Secteur Informel au Tchad (ECOSIT)� surveys in 2011, 2003/2004, and 1995. These surveys covered household consumption and several other important socio-economic characteristics. In addition, INSEED has conducted MICS surveys (sponsored by UNICEF) in 2000 and 2010, and Demographic and Health Surveys (sponsored by USAID) in 1996-1997 and 2004-2005. 2. INSEED, with support from the World Bank’s Trust Fund for Statistical Capacity Building (TFSCB), completed a thorough review of the strengths and weaknesses of the national statistical system. The review highlighted the need to develop improved methods for socioeconomic statistics, with emphasis on agriculture, livestock, fishing, and rural livelihoods. In addition, the review highlighted the need for improved systems of harmonization, standardization, and integration of administrative statistics, related specifically to employment, business and civil registration, education, employment, health, justice, and telecommunications. These systematic reviews were then used to develop a new National Strategy for the Development of Statistics (NSDS), laying an ambitious agenda for new products, including a new round of the household survey in 2018. 3. In parallel there was also a separate project “regional program to harmonize and modernize living conditions surveys (P153702)1� to improve statistical capacity among the eight member states (Benin, Guinea Bissau, Burkina Faso, Côte d’Ivoire, Mali, Niger, Senegal and Togo) of the West African Economic and Monetary Union (WAEMU). While not formally a member of WAEMU (though having observer status), Chad has many similarities, including in terms of language, budget and capacity constraints, the range of agroecological zones, and traditional agriculture and pastoralist practices. It is also one of the five Sahel countries. This project leveraged the ongoing activities of the WAEMU commission to provide technical assistance to INSEED by incorporating Chad into the associated working groups, and by providing partial funding for the new survey. The funding was complemented by additional funds from IDA (Chad Statistical Development project (P159434); US $12,000,000) and contributions from Government. 4. Chad’s Country Partnership Framework2 was discussed by the Board in December 2015 to cover FY16 - FY20 and identified three main themes of engagement: (i) strengthening management of public resources; (ii) improving returns to agriculture and building value chains; and, (iii) building human capital and reducing vulnerability. The CPF further states that it is a priority to assist the Chadian authorities in implementing key statistical projects in the short term while simultaneously building capacity in activities that ensure long term sustainability. Raising the profile of INSEED as the provider of high quality, regular, and necessary statistics encourages long term government financing. At the same time, integrating new technologies and techniques, such as advanced sampling techniques and computer 1 The PDO is to support the WAEMU Commission at strengthening the capacity of its member countries to conduct living conditions surveys that meet harmonized, regional standards and to make the collected micro-data publicly accessible. 2 Report No. 95277-TD Page 4 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) assisted data collection, reduce the per-unit costs and promotes sustainable budgeting as envisaged by the CPF. Project Development Objectives (PDOs) 5. The overall development objective of this project is to strengthen the capacity of the national statistics organization to conduct living standards surveys that meet international quality standards and to publicly release resulting anonymized microdata. Key Expected Outcomes and Outcome Indicators 6. Theory of Change: The Project Appraisal Document (PAD) did not present an explicit Theory of Change (ToC) to guide project design, and therefore the ICR can only reconstruct the logical connections between the project activities and the intended outcomes along with the underlying assumptions. The overall project objective indicates that the project is essentially a statistical capacity building intervention meant to substantially improve the capacity within INSEED through learning-by-doing. The main expected outcomes of the project are the following: (i) improved capacity in INSEED to design, implement, and analysis large multi-topic datasets; (ii) improved quality of data collection though improved training and the incorporation of technology- based data checking; and (iii) release of relevant socioeconomic statistics and public use datasets within 12 months of the completion of fieldwork. To achieve these outcomes, the project supports INSEED with technical assistance to learning-by-doing along the full data production chain from preparatory activities to full analysis and dissemination. 7. The key outcomes of the project will be assessed through the 2 PDO level indicators: (i) Survey preparation is completed according to international standards with regards to quality and timeless. (ii) Relevant socioeconomic statistics and anonymized microdata are released within 12 months of the completion of fieldwork. Components 8. The project was organized around four key components of survey implementation covering preparation, analysis, capacity building, and project management activities. These components are as follows: (i) Preparation activities: these include a CAPI pilot test; the implementation of a national non-standard unit survey covering local harvest and sales measures; the purchase of necessary materials, including tablet computers; publicity and outreach to encourage participation; and class room training for supervisors and enumerators for a period of approximately three weeks, followed by a field test prior to the launch of the survey (ii) Data cleaning, analysis, and dissemination: these include necessary goods to ensure the proper cleaning, analysis, and archiving of data, upgrading headquarters hardware and software; technical assistance through international consultants and two off-site retreats Page 5 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) to support the finalization of the dataset and statistical abstract; translation and printing of the reports, and workshops in N’Djamena and other regional capitals on the results. (iii) Capacity building activities: these include a resident advisor to be embedded in INSEED, participation of INSEED staff in WAEMU technical assistance workshops; Consultant Support for Implementation. (iv) Project management: this includes financial management and procurement personnel with INSEED to successfully implement the survey, and to build capacity for future projects, either through consultants or the hiring of a firm. Restructuring 9. The recipient-executed Trust fund for Statistical Capacity Building (TFSCB - TFA3614) for “Data Production and Capacity Building in Chad� was signed on October 5th, 2016, it became effective on February 15th, 2017. The grant was implemented by the National Statistical Office (INSEED) and the first disbursement was made on April 5th, 2017. 10. During implementation, the project underwent a level 2 restructuring to allow the completion of the remaining activities. Due to project start up difficulties and delays impacted the timeline of the project implementation. The project thus underwent a level 2 restructuring in March 2018. This restructuring was to extend the closing date of the project for 12 months from June 30, 2018 to June 30, 2019. The extension aimed to enable the project to fully implement all planned activities including the production of the statistical abstract (updated poverty numbers, and tables of major correlates of poverty) and dissemination (archived and anonymized datasets released publicly). The restructuring did not have any change in PDO, results framework or procurement. INSEED had prepared an updated action plan that was acceptable to the Bank with a revised implementation schedule and the disbursement estimates had been changed. 11. At the time of the request of restructuring, all the preparatory and most capacity building activities were completed. The questionnaire had been revised following the best practice methodology developed as part of the West African Economic and Monetary Union (WAEMU) project P153702. Cluster size was substantially reduced, to the WAEMU guideline of approximately 12 households per cluster. The non- standard unit survey was completed in December 2017 and the pilot survey was completed in February 2018. Field work for the first wave was scheduled to begin mid-May 2018. II. OUTCOME Relevance of the PDO and Rating 12. The CPF states that it is a priority to assist the Chadian authorities in implementing key statistical projects in the short term while simultaneously building capacity in activities that ensure long term sustainability. Raising the profile of INSEED as the provider of high quality, regular, and necessary statistics encourages long term government financing. At the same time, integrating new technologies and techniques, such as Page 6 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) advanced sampling techniques and computer assisted data collection, reduce the per-unit costs and promotes sustainable budgeting. 13. The project objective is highly relevant given the country context, government policies, and World Bank Group strategy. The objective of delivering reliable and timely statistics in accordance with international standards and best practices is consistent with the Government’s priorities and the World Bank Group’s Country Partnership Framework (CPF), both at appraisal and closing. High quality and accurate data are required to both assist the public sector in targeting and monitoring its interventions and to provide the private sector with enough information to make effective investment decisions. With the help of the project, and for the first time since 2011, INSEED has produced a high-quality data relevant for government policies. 14. Therefore, the relevance of the PDO is rated high. Achievement of PDO (Efficacy) Assessment of Achievement of Each Objective/Outcome 15. Based on the PDO statement, there are two PDO level indicators and three intended outcomes. PDO Indicator One: Survey preparation is completed according to international standards with regards to quality and timeless. 16. In order to successfully implement the survey, Chad decided to participate in the WAEMU household survey harmonization project. In fact, the country adjusted its survey calendar to match that of the WAEMU project, which led to a delay in starting the activities. In addition, difficulties with procurement resulted in additional delays in the implementation of fieldwork. Once it started, the survey implementation followed the updated timeline. The training of enumerators for the non-standard unit survey took place from June 20th to June 27th, 2017 and the data were collected over a two-month period as planned from October 8, 2017 to December 05, 2017. Data cleaning, estimation of conversion factors for non-standard units as well as the photo album of non-standard units were completed 45 days after the end of the NSU data collection. The household survey data were collected during two waves of three months each (from 18 June to 18 September 2018 for the first wave and from 15 January to 15 April 2019 for the second wave). The duration of 4 months between the two waves is consistent with the methodology that requires a time lag between the two waves to identify seasonality effects, to retrain enumerator and to prepare the data collection of the second wave. The use of CAPI for data collection eliminated the data entry phase which was one of the main causes of delays in the previous surveys. Two weeks after the end of the data collection, a mission of the world bank worked with INSEED team to clean the data over a period of two weeks (from April 20 to May 5, 2019). Additional data cleaning workshops were completed by the INSEED and their participation to regional workshop organized by the WAEMU project allow them to improve the quality of the data. Currently, Chad data is considered as cleaned and available even if the team is continuing to improve the quality of data with additional cleaning workshops. Page 7 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) 17. The survey was implemented in a high-quality manner. The delay in starting fieldwork due to readjustment of the timeline with that of WAEMU project allowed INSEED to benefit from all the technical assistance and the technical document developed for the WAEMU project. The questionnaires were adapted to the Chadian context during workshop in Ndjamena with the participation of staffs from INSEED, ministries (education, health, agriculture, etc.), African Development Bank and World Bank. Data were collected during two waves to take into account seasonality in consumption and to better estimate household poverty. In the previous surveys, INSEED collected expenditures data to estimate poverty while it is well known that consumption is the best data to measure poverty in African countries. In this survey, the country followed this requirement and collected both expenditures and consumption data for poverty measurement. Finally, to ensure high quality data, the survey was conducted using electronic questionnaires developed on Survey Solution. This allowed for multiple levels of data quality control including enumerators, supervisors and dedicated technical team at headquarter. A program of data quality checking developed by the World Bank team were adapted and used by INSEED to check/correct/validate the quality of data in a timely manner. 18. The system of data verification and checking developed by the INSEED was robust with many levels of control. The first level of control was the data collection method which was based on CAPI. Indeed, the control of quality, exhaustivity and likelihood of data integrated in the electronic questionnaire allow to do a first verification of data during the data entry on the field. After this first verification, data collected by enumerators were sent to supervisors who systematically check and decide either to validate or reject them. Rejected data are sent back to enumerators for checking while validated data are sent to the technical team members who also check the data and decide either to validate or reject them. Rejected data are sent back to supervisors while validate data are sent into the server. Every week, the Headquarter run a data checking program on the data available on the server. This dofiles had been developed by the world Bank staffs and adapted by INSEED team. It allowed the team to generate basic statistics on data for comparison with the results of the previous surveys. It also allowed to monitor the performance of each enumerator and to identify those responsible for erroneous data for action to be taken to correct these data. The data quality reports were shared with the World bank team for advice with the aim to better assess the quality of the data in a timely manner. 19. INSEED carried out close supervision missions in the field in order to better carry out data checking and ensure the smooth running of field operations. Six supervisions teams were created to better cover the survey sample across the country, and each team made three missions of 14 days/mission during each wave (42 days of supervision over 90 days of collection during each wave). The first purpose of these missions was to check the quality of interview with respect to the guidelines. Secondly, these missions aimed to verify if interviewed households were those selected as part of the sample and if all the questionnaire was covered during their interview. Finally, these missions helped to resolve technical and administrative constraints to the data collection. 20. Given the low coverage of internet in the country, the technical team set up a WhatsApp group to better manage communication within all the survey team, including enumerators, supervisors, technical team member at headquarter. Thus, the global instructions to be provided to all members of the survey were shared on this group in addition to direct communications with fields teams. The WhatsApp messages or audio recording allowed teams which were working in non-network covered areas to get access to Page 8 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) information on the survey as soon as they moved to areas covered by internet. This strengthened the communication within team and improved the quality of data collection. PDO Indicator Two: Relevant socioeconomic statistics and anonymized microdata are released within 12 months of the completion of fieldwork. 21. INSEED has already elaborated the survey report analysis plan and the staffs who will contribute to the report have been identified. INSEED has a plan to organize a workshop in to write the survey report analysis. This report will include all sections of the survey with the exception of the section on poverty. The poverty section will be written jointly by INSEED and the World Bank. INSEED is planning to have the whole report completed by end of March 2020. A national meeting to present the report findings is scheduled for end of March 2020 and the report dissemination process will start at the same with the funding of the Chad Statistical Development P159434 of the statistical project. 22. INSEED has adapted the methodology developed by the World Bank as part of the WAEMU survey harmonization project to calculate its consumption aggregates. All the technical decisions taken by INSEED in the implementation of this methodology have been documented and the methodological note is available. Outcome One: Improved capacity in INSEED to design, implement, and analysis large multi-topic datasets. 23. The capacity of the survey team in design and implementation of large-scale surveys has been strengthened. Indeed, the team benefited from capacity building during different technical workshops organized at the regional level and at the national level with the technical assistance of the World Bank and the WAEMU commission. The resident advisor support during the first wave was also helpful in building the team capacity (survey design, implementation and management, training, data checking). These supports allow the team to become independent. They were able to lead and implement the second wave of the survey without any support of a resident advisor. 24. INSEED has also acquired new capacities in survey implementation based on the use of CAPI. Indeed, this survey is the first CAPI-based data collection implemented by the team in charge of households living conditions surveys in INSEED. The CAPI trainings carried out as part of the harmonized survey project contributed to strengthening the capacity of INSEED in CAPI survey design. Three staffs have benefited from CAPI training and are able to design a survey using this technology. These staffs trained the other team members and at least 6 members of the survey team are now able to design a CAPI-based survey. 25. INSEED has strengthened its capacity in large-scale survey data processing thanks to its participation in the regional workshops organized by the WAEMU project. The data cleaning and analysis as well as the calculation of the consumption aggregates have been carried out by INSEED. In addition, the regional poverty analysis workshops held in Cotonou and Ouagadougou enabled the INSEED team to strengthen its analytical capacity. In total, 5 staffs benefited from these capacity enhancements in data analysis. Outcome Two: Improved quality of data collection though improved training and the incorporation of technology-based data checking. Page 9 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) 26. The data quality control system sets up by INSEED has led to an improvement in the quality of the data. Compared to the previous surveys, this system integrated a headquarter technical team in charge of data checking while the data collection was ongoing. This team worked on validating the data approved by supervisors. This improve the quality of data and decrease the duration of the data cleaning process. 27. The data quality has also been improved thanks to better training of enumerators. INSEED was supported during the enumerators’ training by the resident advisor and the World Bank's international consultants. In addition, the duration of the training was relatively longer (one month) and included several practical exercise sessions for the administration of the questionnaires. The best enumerators were selected based on a test. Outcome Three: Release of relevant socioeconomic statistics and public use datasets within 12 months of the completion of fieldwork. 28. INSEED has already begun the analysis of the survey report. This report will be available by the end of February 2020 and its dissemination will be effective from this period according to the INSEED calendar. The institute also plans to make available to the general public metadata of main monitoring indicators of the National Development Plan. The data will ultimately be used to write the Poverty Assessment, SCD, Gender Assessment, etc. 29. INSEED is currently creating a portal for the publication of micro-data of the survey. A team from INSEED is on mission in Senegal to learn from the experience of Senegal in term of data portal management. Following this mission, INSEED will finalize its portal and will release some of the data on the portal by January 2020 for a pilot phase. The entire data of the survey will be released for the public use before February 2020, approximately 10 months after the end of the data collection. 30. To ensure wide dissemination of data, INSEED and the Bank will discuss the conditions under which data can also be made available to the general public through the World Bank website. This agreement will take into account the data dissemination policy as defined in the new statistical law adopted in the country in 2019. 31. The project almost fully achieved its two PDO level objectives and three intended outcomes. The overall Efficacy of the project is rated substantial. Efficiency of PDO 32. At appraisal, it was recognized that the project does not lend itself to conventional economic cost-benefit analysis or financial analysis due to the public good nature of its output (a survey). As public goods, official statistics are generally financed from Government revenue, and from partners and donors’ contribution. Therefore, no financial return was expected from this project. However, the project outcomes are expected to promote improved allocation of resources by enhancing the monitoring of the impact of Government spending by providing more accurate information on socioeconomic activity to the public at large. 33. The total cost of the survey was financed by the 1.5 million from TFSCB and complemented with 2.0 million financing from IDA. The survey covered 7,000 households which gives a per household average cost of US$500 compared to US$300 in Niger, a similar Sahel country. The costs involved in achieving Page 10 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) the operation’s laudable objectives/outcomes were not reasonable in comparison with both benefits and value for money. Transportation and communication costs were much higher in Chad compare to Niger. 34. Given some of implementation difficulties that delayed field work and the unreasonably high cost of the survey, the efficiency of the project is rated moderate. Overall Outcome Rating 35. Based on the combined assessment of project relevance (high), efficacy (substantial) and efficiency (moderate), the Project is rated Moderately Satisfactory (MS) in achieving its PDO “To strengthen the capacity of the national statistics organization to conduct living standards surveys that meet international quality standards and to publicly release resulting anonymized microdata.� This is justified as the project has met one out of the two key PDO indicators. The result “Relevant socioeconomic statistics and anonymized microdata are released within 12 months of the completion of fieldwork.� hasn’t happened yet, but the team is confident this will take place as well within the stipulated timeline. Indeed, INSEED is currently creating a portal for the publication of micro-data of the survey and plan to release the survey data for the public use before February 2020, approximately 10 months after the end of the data collection. Other Outcomes and Impacts 36. The portal in preparation with the aim to release the survey data for public use will go beyond this survey. Indeed, INSEED is taking advantage for this to improve its dissemination strategy. Many other surveys data available in the institute as well as the incoming other surveys will be released for public use in the future through this portal. In addition to micro-data, INSEED plans to set up an online platform based on which the public will be able to make generate some indicators and graphs based on data available on the portal. All these activities have been funded by the project. The World Bank staff in charge of the project have also provided technical assistance and guidance for achieving these activities. This is a great institutional strengthening of the Chadian statistical system as it will allow INSEED and the Chadian statistical system to better disseminate data and statistical reports and to increase knowledge of and use of the data and the survey report. 37. The project funded the participation of INSEED to many capacity building activities including survey design and management, sampling, data checking and cleaning, data analysis organized by the WAEMU and The World Bank under the “regional program to harmonize and modernize living conditions surveys (P153702)�. INSEED also benefited from technical discussions and share of experiences between all the countries participating to this regional program. These allow INSEED to improve its capacity in term of collaboration with other statistical offices. 38. INSEED is currently working to change the national account base year from 2005 to 2017. The preliminary data of the survey supported by this Project have been used for this purpose. Page 11 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME 39. The procurement procedure in Chad has had an impact on the start of the preparatory work. Indeed, public procurements over 10,000,000 FCFA (about US$ 20,000) require the approval of the Presidency of the Republic before execution. The procurement of the survey materials (tablets and others) was above the threshold and therefore should follow this procedure. This resulted in extending the purchase period of the tablets. In addition, tablets initially supplied did not meet the recommended technical standards for CAPI data collection. As a result, INSEED demanded the replacement of these tablets by the supplier, which resulted in additional delays. All these issues related to procurement delayed the starting of the survey by about 8 months. 40. The lack of internet coverage in some areas of the sample has been a handicap for data collection and checking. In fact, the data collection teams who were working in these areas were obliged either to travel long distances to transfer the data to the servers or to postpone the transfer of the data to their movement to other locations covered by the network. This had the effect of delaying data transfer to the server and indeed the duration of the data checking process. 41. The gap between the initial programming of the Trust Fund activities and the WAEMU project activities has led to a delay in the execution of activities. Indeed, activities such as participation in sub-regional workshops for the finalization of technical documents, data cleaning and treatment, calculation of consumption aggregates, etc. have been delayed from their initial programming. 42. The headquarter team of INSEED did not have a dedicated office. They used to use the meeting room for their works. Thus, the team had to stop work each time a meeting was scheduled in the room. 43. Security issues disrupted the activities in certain areas. Indeed, in the Lake Chad and North East regions, that have experienced repeated attacks, INSEED has requested and benefited from military escort to ensure the safety of enumerators. This situation has generated costs related to escort support but has also delayed data collection. Similarly, the security constraints also forced the INSEED team to replace enumeration areas, which necessitated readjustments in the execution of the data collection. 44. The isolation (due to both security and weather factors) of certain enumeration areas has also been a major constraint in carrying out data collection. In order to fix this problem, INSEED has redistributed the enumeration areas between the two waves so that data collection can proceed rapidly in areas inaccessible during the rainy season. For enumeration areas located on the islands in Lake Chad, INSEED had to pay for the services of motorized boats to move the teams. This situation has affected the transport budget. 45. The obsolescence of census cartography (2009) has delayed the enumeration work. INSEED updated the cartography of the enumeration areas of the sample. This caused a Page 12 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) delay in the start of enumeration activities and consequently the data collection. This new cartography also generated additional costs in the budget of the survey. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME Quality at Entry 46. The project design at entry is considered Satisfactory. It was based on a thorough review of the strengths and weaknesses in the statistical system which accurately identified the constraints limiting the ability of the system to meet stakeholders needs for high quality statistics. The technical aspects at entry were well addressed by the project including taking into account lessons learned from survey design and implementation in similar operations for other countries such as Sierra Leone and Liberia. Quality of supervision 47. Overall, Bank performance was strong although procurement procedure in Chad delayed implementation. Once this issue was resolved, the project implementation followed the timeline and two of three results indicators were achieved and the remaining one will also be achieved in accordance with the deadline. 48. INSEED received strong implementation support funded by the TF and also by the IDA-financed project. A Resident Technical advisor hired by the project was based in Ndjamena for one year working on daily basis with INSEED’s team for the implementation of the survey. Bank staff and international consultant provided technical assistance to the Institute during the entire implementation of the survey. INSSED participated in international workshops organized by the Bank which improved its capacity in term of survey design, implementation and treatment. Bank staff from the Chad Country Office also provided support on all new Bank processes and procedures. Overall Rating of Bank Performance 49. Based on the justifications of both quality at entry and quality of supervision, the overall Bank performance is rated Satisfactory. Risk to Development Outcome 50. The ECOSIT series is the core of INSEED’s statistical calendar and will undoubtedly continue for many years to come. However, sustainable financing remains a risk but with support from World Bank and other donors, this risk is mitigated in the medium term in particular by another planned ECOSIT5 financed by IDA. In the longer term, it is hoped that an increased exposure associated with regular delivery of high-quality data will cultivate domestic demand for data and the main funding source can be shifted to government sources. Page 13 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) V. LESSONS LEARNED AND RECOMMENDATIONS 51. INSEED needs increased capacity in procurement and financial management in order to satisfactorily implement projects. Without a strong understanding of fiduciary aspects both of World Bank and government procedures, it is difficult to implement project activities in a timely manner. 52. More broadly, INSEED needs increased capacity in management of survey activities. Weakness of communication within PIU (technical coordinator, procurement specialist and financial specialist) due to lack of coordination meetings hampered the implementation of activities. In addition, the fact that the team was involved in other activities did not allow them to most effectively implement the project activities. 53. The process of enumerators’ selection should be made even more rigorous and should consider gender aspect. Indeed, number of enumerators did not have experience in CAPI- based data collection. Therefore, the duration of the training was extended in order to fix these issues that could have been avoided with strict respect of selection criteria (logic/basic math and survey completion skills, experience in technology-based data collection, multi-language, etc.). In addition, while there were some female enumerators in the household survey, the non-standard unit survey has been implemented by only male enumerators. 54. The data collection headquarter at INSEED was effective but should be improved. The number of staffs working to check the data in a very timely manner in the headquarter was low. Consequently, feedbacks on data quality and correction for given enumeration areas were sent to field teams when they had already left the concerned enumeration areas. In addition, there is a need of dedicated office/room for headquarter as data checking require high level of concentration. The team was disturbed each time a meeting was planned in the meeting room where the headquarter was based. 55. INSEED needs increased capacity in data cleaning and treatment and poverty analysis. The INSEED team participated in the regional capacity building workshops and benefited from capacity building during World Bank mission in Chad. In addition, the poverty analysis will be done by the team with support from the World Bank. This will increase their capacity in poverty analysis. Nevertheless, the team capacity should be reinforced in term of data treatment and analysis, especially for the young team members. . Page 14 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Survey preparation is Yes/No N Y Y Y completed according to international standards with 01-Jul-2016 01-Jul-2017 01-Dec-2017 30-Jun-2019 regards to quality and timeliness. Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Relevant socioeconomic Yes/No N Y Y N statistics and anonymized microdata are released within 01-Jul-2016 01-Jul-2017 01-Mar-2018 01-Nov-2019 12 months of the completion of Page 15 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) fieldwork. Comments (achievements against targets): A.2 Intermediate Results Indicators Component: Preparation activities, data cleaning and dissemination, and capacity building Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Finalization and testing of Yes/No N Y Y Y Computer Assisted Personal Interviewing (CAPI) data entry 01-Jul-2016 01-Dec-2016 01-Mar-2017 28-Jun-2019 system Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Non-standard unit survey Yes/No N Y Y Y conducted 01-Jul-2016 01-Dec-2016 01-Mar-2017 28-Jun-2019 Comments (achievements against targets): Page 16 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Generation of key consumption Yes/No N Y Y Y statistics 01-Jul-2016 15-Feb-2017 30-Nov-2017 01-Jul-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Survey enumerator training Yes/No N Y Y Y completed 01-Jul-2016 11-Oct-2016 01-Dec-2016 22-May-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of regional workshops Number 2.00 6.00 6.00 6.00 hosted by WAEMU with participation of INSEED staff 01-Jul-2016 06-Jun-2017 01-Dec-2017 02-Dec-2019 Comments (achievements against targets): Page 17 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Statistical abstract is published Yes/No N Y Y N within 12 months of completion of field work and 01-Jul-2016 29-Nov-2017 01-Mar-2018 20-Aug-2019 contains gender disaggregated analysis where relevant Comments (achievements against targets): Page 18 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) B. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1 1. Survey preparation is completed according to international PDO Indicator 1 standards with regards to quality and timeliness 1. Finalization and testing of Computer Assisted Personal Interviewing (CAPI) data entry system 2. Non-standard unit survey conducted Intermediate Results Indicators 3. Survey enumerator training completed 4. Number of regional workshops hosted by WAEMU with participation of INSEED staff 1. Improved capacity in INSEED to design, implement, and analysis Key Outputs by Component large multi-topic datasets (linked to the achievement of the Objective/Outcome 1) 2. Improved quality of data collection though improved training and the incorporation of technology-based data checking Objective/Outcome 2 1. Relevant socioeconomic statistics and anonymized microdata are PDO Indicator 2 released within 12 months of the completion of fieldwork. 1. Statistical abstract is published within 12 months of completion of Intermediate Results Indicators field work and contains gender disaggregated analysis where relevant Key Outputs by Component (linked to the achievement of the Objective/Outcome 2) Page 19 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) . ANNEX 2. PROJECT COST BY COMPONENT Actual after Amount at Amount at restructuration Components Approval Closing Project Closing (US$M) (US$M) (US$M) Preparation activities 645,255 645,255 776,159 Data cleaning, analysis, and dissemination 297,562 297,562 146,907 Capacity building activities 323,545 323,545 324,545 Project management 200,000 200,000 196,730 Total 33,638 33,638 52,635 Total 1,500,000 1,500,000 1,496,977 Page 20 of 21 The World Bank Data Production and Capacity Building in Chad (P159189) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Data production and Capacity Building Trust Fund provided to INSEED in the context of the implementation of ECOSIT4 has been a considerable contribution. Below are some major points that show to what extend the trust fund contributed to the system. In terms of capacity building, the trust fund made it possible to train several statisticians in charge of the survey through their participation in regional workshops in survey management. These include training in sampling, CAPI-based survey design, data processing, etc. This funding also allows INSEED to acquire highly efficient work equipment including computers, tablets and other equipment. Today, a critical number of staffs are trained making INSEED able to implement complex surveys similar to ECOSIT 4 without support from external expertise. It should also be noted that these staffs have been providing substantial support in the use of technologies to implement survey to other institutions in the country. Overall, the Trust Fund brought oxygen to INSEED and allowed the institution to comply and be almost at the same level as other National statistical offices through improving staffs’ capacities in term of use of technology to implement household surveys. Page 21 of 21