Document of The World Bank Report No.: 62874 PERFORMANCE ASSESSMENT REVIEW OF INVESTMENT CLIMATE ASSESSMENTS IN FIVE TRANSFORMING ECONOMIES: BANGLADESH, EGYPT, GUATEMALA, KENYA AND VIETNAM June 30, 2011 Country, Corporate & Global Evaluation Independent Evaluation Group Abbreviations and Acronyms AAA Analytic and Advisory Activities ACS Activity Completion Summary CAE Country Assistance Evaluation CAS Country Assistance Strategy CD Country Director CEM Country Economic Memorandum CGE Computable General Equilibrium CM Country Manager CODE Committee of Development Effectiveness DB Doing Business DEC Development Economics Unit (World Bank) EPZ Export Processing Zone ESW Economic and Sector Work EU European Union FDI Foreign Direct Investment FPD Finance and Private Sector Development GDP Gross Domestic Product ICA Investment Climate Assessment ICR Implementation Completion Report ICT Information and Communication Technology IDA International Development Association IEG Independent Evaluation Group IEGCC IEG Country, Corporate and Global Evaluation IMF International Monetary Fund NGO Non-governmental Organization PAR Performance Assessment Review PER Public Expenditure Review PPAR Project Performance Assessment Report PREM Poverty Reduction and Economic Management QAG Quality Assurance Group SM Sector Manager SME Small and Medium-sized Enterprises TA Technical Assistance TFP Total Factor Productivity TTL Task Team Leader WBG World Bank Group Director-General, Independent Evaluation : Mr. Vinod Thomas Senior Manager, IEG Country, Corporate & Global Evaluation (IEGCC) : Mr. Ali M. Khadr Task Manager : Mr. Chad Leechor iii Contents Preface................................................................................................................................. v Summary ........................................................................................................................... vii 1. Introduction ..................................................................................................................... 1 2. ICAs as an Instrument of the World Bank Group........................................................... 5 3. The ICA Experience in the Five Countries ................................................................... 11 4. Results of the ICAs ....................................................................................................... 17 5. Strategic Relevance of the ICAs ................................................................................... 23 6. Quality of the ICAs ....................................................................................................... 29 7. Dialogue and Dissemination ......................................................................................... 35 8. Findings, Issues and Suggested Approaches ................................................................ 39 Boxes Box 1. Bangladesh: The Power and Limitations of ICAs ................................................ 13 Box 2. Guatemala: The Tale of Two ICAs ....................................................................... 18 Box 3. The Climate for Investment or for Operations? .................................................... 30 Tables Table 1: Rating Criteria ...................................................................................................... 3 Table 2: Planned and Actual Costs of Selected Investment Climate Assessments ............ 9 Table 3. Time for Completion - Selected Investment Climate Assessments ...................... 9 Table 4. The Core Investment Climate Constraints Identified In ICAs ........................... 11 Table 5. Results Ratings ................................................................................................... 21 Table 6. Ratings on Relevance.......................................................................................... 26 Table 7. Ratings for Technical Quality ............................................................................. 32 Table 8. Ratings for Dialogue and Dissemination ............................................................ 38 Figure Figure 1. Number of ICAs Delivered, FY03-FY11 ............................................................ 7 iv Annexes ANNEX 1: BANGLADESH ............................................................................................ 45 ANNEX 2: EGYPT........................................................................................................... 61 ANNEX 3: GUATEMALA .............................................................................................. 75 ANNEX 4: KENYA ......................................................................................................... 93 ANNEX 5: VIETNAM ................................................................................................... 117 ANNEX 6: LIST OF PEOPLE MET .............................................................................. 125 v Preface The IEG evaluation team wishes to thank country-unit staff, public officials, researchers and representatives of the private sector in Bangladesh, Egypt, Guatemala, Kenya and Vietnam for the time, insights and logistic support provided during the team‘s field visit. In particular, the team gratefully acknowledges the assistance provided by Laura Frigenti, Ellen A. Goldstein, Victoria Kwakwa, A. David Craig, and Johannes Zutt. The report benefited from the comments of Kris Hallberg and Mary Hallward-Driemeier, who served as Panel Reviewers, as well as those of Martha Ainsworth, Ismail Arslan, Amit Banerjee and Mark Sundberg. The evaluation team comprises: Chad Leechor (task team leader), Basil Kavalsky, Tim De Vaan, Jorge Garcia-Garcia and Mauricio Carrizosa (consultants). Vikki Taaka provided research and administrative support. The work was carried out during the period from October 2010 to April 2011 under the supervision of Ali Khadr. vii Summary 1. The Bank‘s Investment Climate Assessments (ICAs) were introduced in 2002. At that time, the Bank‘s analytic work was increasingly demonstrating that countries with open economies and market-friendly policy frameworks were more successful in achieving growth and poverty reduction. The ICA includes a diagnostic survey of the business community to identify the key constraints to private sector investment and operations. The survey provides the basis for a report by the Bank that analyzes the evolution of the investment climate over time and provides comparisons with other countries. The report provides recommendations on steps to improve the investment climate that are intended to inform the policy dialogue and the design of follow-up Bank lending and analytic work. The ICA is repeated at 3 to 6 year intervals to monitor the progress on improving the investment climate. 2. This Performance Assessment Review (PAR) is part of a new IEG initiative to assess the impact of Bank analytic work. The PAR series takes both a due diligence perspective of whether the client and the Bank are deriving benefits commensurate with the substantial cost of this work, and a learning perspective of deriving lessons to inform the design and implementation of analytic work in the future. It is important to clarify, however, that a PAR has narrower and more targeted objectives than those of IEG‘s thematic or sector evaluations. It concentrates on the selected reports under review and the findings are not necessarily applicable to the entire product line. This particular PAR focuses on ICAs of five selected countries. It does not address broader issues pertaining to the merits of the ICA or enterprise survey generally. 3. Since its introduction and piloting, the ICA has become an important knowledge product. The WBG has produced more than 100 ICAs covering more than 90 countries across all regions. In more than 20 countries, two or more ICAs have been produced. For this review, IEG identified 5 countries which broadly fit the definition of transforming economies – those at a critical stage of moving to a second generation of more sophisticated manufacturing and service sector activities.1 These are: Bangladesh, Egypt, Guatemala, Kenya and Vietnam. Nine ICAs have been prepared for these countries, one in Vietnam and two in each of the others. A short mission was undertaken to each country to meet with government officials, private businesses and Bank staff. 4. The enterprise surveys for the ICAs are based on a standardized methodology that has been developed and is managed by a central unit in the Finance and Private Sector Development Network. This practice allows for the publication of comparable cross- country data for inclusion in the reports. Beyond the base of standard questions, there are additional questions that are customized to the circumstances of individual countries. This allows for some variation in the scope of ICAs, ranging most narrowly from surveys of formal sector manufacturing enterprises with more than 10 employees in two or three 1 According to the 2008 World Development Report, ―transforming economies‖ are characterized by rapidly rising rural-urban income disparities which generate social and political tensions. Agriculture has a declining share in the economy and is no longer a major source of growth, but poverty remains primarily a rural phenomenon (with rural residents accounting for more than 80 percent of all poor). viii of the largest cities, to surveys that include small informal enterprises in both services and manufacturing and cover both urban and rural areas. In most countries the pattern has been that the first ICA starts with a fairly narrow sample, but subsequent ICAs have expanded coverage reflecting the interest of the Government and the Bank in comparing the constraints faced by different sub-sets of enterprises (urban vs. rural; small vs. large; manufacturing vs. services.) 5. The surveys in the five countries include a set of quantitative questions on business results to enable the analysts to derive data on the evolution of total factor productivity. The core of the survey however, consists of questions which ask the owner or manager to assess the importance of a range of constraints covering most features of the enabling environment: macro-economic conditions; security and crime; governance issues, particularly informal payments; licensing and regulation; tax levels and administration; availability of infrastructure, particularly energy and transport; access to land; access to finance; and availability of skilled labor. The survey is generally preceded by a large launch workshop to explain to the Government and members of the business community the objectives and methodology. The actual conduct of the survey is contracted to a local academic institute, think-tank or consulting firm with technical support provided by experts from FPD. A team is constituted either from the PREM network or FPD to supervise the implementation of the survey and prepare the analysis. Once the raw data has been tabulated, the team prepares a report identifying the key constraints and tracking their evolution over time (if baseline data is available), and comparing them with the results for a selected group of 5 to 10 countries. 6. The enterprise survey is essentially a diagnostic tool. It identifies what the constraints are, but says little about what can or should be done to relieve them. Early ICA reports were very much focused on simply publicizing the business community‘s perceptions of the problems. Both Governments and Bank managers however, reacted adversely to reports confined to diagnosis with no advice as to treatment. As a result almost all ICAs now have a matrix of recommendations as to what steps should be taken to deal with the problems, often divided into short-term and longer-term responses. This creates a dilemma, however, since the constraints, ranging from broad general issues (e.g. the level of inflation), to highly specialized ones (e.g. environmental, health and safety regulations) require in-depth analysis in order to go beyond general statements. This is a central issue in the ICA approach that has not been adequately addressed. 7. The nine ICAs selected for this PAR have been evaluated following a new IEG methodology. The four criteria that have been applied are:  Results – to what extent recommendations were adopted or influenced behavior;  Relevance – whether the product was consistent with client and Bank priorities;  Technical quality – what types of analysis and policy advice were provided;  Dialogue – whether the goals and outcomes were shared with the audiences. 8. There was a wide variation in the results of the ICAs. The Egypt 2004 ICA was rated Highly Satisfactory given the use of the ICA findings by the Ministry of Investments as an agenda for reform. It was also used as a baseline for monitoring the degree to which Government measures on taxation and deregulation were reflected in the ix perceptions of the business sector. By contrast, the Guatemala 2008 ICA and the Vietnam 2006 ICA were rated less than Satisfactory. In the case of Guatemala the ICA was essentially ignored by a Government whose priorities were on social issues rather than promoting private investment. The preparation of the Vietnam ICA was delayed and when it was finally available it was viewed as already out of date by the Government. It is evident that the results of ICAs relate very closely to the ownership and commitment of the Government. This is not a matter of ownership of the ICA report. On the contrary, the evaluation finds that much of the credibility of the ICA stems from the fact that it is not a Government sponsored product. Government ownership refers to acknowledgement of the constraints faced by business and the ownership of the agenda for addressing these constraints. 9. Two ICAs were rated Highly Satisfactory on relevance. The Egypt 2004 ICA was ideally timed to capitalize on the commitment of the authorities to raising the level of private investment, which was the subject of a newly created ministry. The Guatemala 2004 ICA was similarly well timed to take advantage of the Government‘s interest in pursuing a range of reforms to support private investment. By contrast the ICA that was rated less than satisfactory was poorly timed in relation to the interest in taking up this agenda. The Bangladesh 2008 ICA was associated with an interim Government and appears to have been ignored if not rejected by the successor Government because of this association. The Guatemala 2008 ICA and the Vietnam 2006 ICA, both rated moderately satisfactory, were of little interest to their respective Governments at the time they were delivered. The Vietnam case also makes clear the importance of timing and the value of interim briefings of preliminary results rather than waiting until the final report is ready. While it is unlikely that the very highest ratings could be achieved in the absence of a ‗champion‘, satisfactory outcomes are possible if the Bank engages a broad range of stakeholders and ensures that the timing is appropriate. The Kenya 2008 ICA which was rated Satisfactory is a good example. 10. In general the ICAs are up to the Bank‘s high standards of technical quality. All except two ICAs were rated satisfactory. In all cases the survey results gave clear diagnostics and the reports provided good analysis of the core constraints to private investment. The two areas of relative weakness were, first, presentation and, second, the quality of the recommendations. On presentation, some reports are confusing with too many comparisons, often using different scales (cross-country and inter-temporal) and pointing in different directions. As the ICA cycle progresses in most countries, the reports also seem to have larger coverage and become longer. The Bangladesh 2008 ICA was rated Moderately Satisfactory, because despite the quality of the analysis, it covers too much and is sometimes confusing as a consequence. 11. The quality of the recommendations is cited as an issue in most of the case studies. As discussed above this is a central dilemma for the ICA. The survey yields diagnostic information, but does not provide any analysis of the likely impact of alternative remedial measures. This has to be drawn from the sector analysis that the Bank carries out in the relevant areas. It would be frustrating if all the ICA can say is that the Government needs to address the energy problem. On the other hand, specific recommendations generally require that the Bank carries out more detailed sector work. Even where the ICA simply takes recommendations from existing Bank sector work it is x difficult to provide sufficient institutional and operational guidance to make them convincing. The Vietnam 2006 ICA was rated Moderately Satisfactory on technical quality inter alia because of the lack of prioritization of its recommendations. 12. With some exceptions, the dialogue associated with ICAs has been conducted well and served the purpose of knowledge creation and sharing. Nevertheless, this is a relatively weak area. Four ICAs were rated satisfactory on this: the Bangladesh 2003 and 2008 ICAs, the Guatemala 2004 ICA and the Kenya 2008 ICA. In all these cases there was a carefully planned process, usually with professional facilitation, wide media coverage, and participation by a broad range of stakeholders. In three cases the ICA was not made public.2 The Egypt 2004 and 2009 ICAs were rated moderately unsatisfactory on this account in spite of a very good process of discussion and broad dissemination within the national government. The Vietnam 2006 ICA was rated unsatisfactory because there was little discussion of the final report. Similarly in spite of being publicly available the Guatemala 2008 ICA was rated Moderately Unsatisfactory on this dimension, because the only discussion held was with the key local partners. 13. The evaluation yielded a number of findings some of which merit highlighting:  The ICA should be viewed as a diagnostic tool that provides an evidence basis for the identification of key constraints to private investment. It should only provide recommendations in areas where proper sector analysis has been undertaken. For the rest the recommendations should take the form of defining the roadmap needed to develop the specific proposals for implementation.  The ambition of the ICA should be tailored to the priority of its sponsors in the Government. On a selective basis it will be appropriate to build on the diagnostics and utilize the ICA as a strategic instrument. This will depend in large part on the identification of a ‗champion‘ who is interested and willing to take the leadership in sponsoring the agenda of reform. In the absence of such a champion or when political transition is imminent, the enterprise survey, rather than the ICA, should be the default option.  The WBG should insist on making the ICA publicly available and expanding the availability of the dataset to researchers and academics in the country. Over time, local researchers should be engaged to participate more actively in the preparation and dissemination of the report. Vinod Thomas Director-General Evaluation 2 Nonetheless, results of the enterprise survey were made publically available on the World Bank‘s website. 1 “We didn’t learn anything new from the ICA. But it provided an evidence basis for what we knew. It was no longer possible for those who opposed reforms to argue that this was just the World Bank’s ideas. We had a survey of their own businessmen to draw on.” (A World Bank Country Director) 1. Introduction 1.1 This Performance Assessment Review (PAR) is part of a new IEG initiative designed to assess analytical and advisory activities (AAAs). The focus of this PAR is a cluster of Investment Climate Assessments (ICAs). A previous PAR reviewed a sample of growth diagnostic reports drawn from the Africa Region. The methodology used in the PAR series, as discussed below, has been developed recently and is still undergoing refinements. 1.2 The Investment Climate Assessment is a major knowledge product of the World Bank Group (WBG), with more than 100 reports produced since FY03, covering more than 90 countries in all Regions. A key objective of ICAs is to identify binding constraints on private investment and growth, thereby motivating the authorities to undertake reforms. It was introduced in 2002 at a time when the Bank‘s analytic work was increasingly demonstrating that countries with open economies and market-friendly policies were more successful in achieving growth and poverty reduction. Objectives 1.3 The objective of this review is to give an independent assessment of the strengths and weaknesses of ICAs, with a view to learning from the breadth and scope of this major activity. It would give an opportunity for the WBG to see what has worked well and what has not. The review would also serve to broaden IEG‘s experience and repertoire of instruments for evaluating AAA products. 1.4 It is important to clarify that the PAR series has narrower and more targeted objectives than those of IEG‘s thematic or sector evaluations. A PAR concentrates on the results, relevance, quality and impact of the selected products under review. A PAR does not address the broader issues pertaining to the merits of the entire product line or the relative value of one analytical instrument in comparison with another, and the findings are not necessarily applicable to ICAs as a whole. 1.5 For this PAR, IEG identified countries which broadly fit the definition of transforming economies: Those at a critical stage of moving to a second generation of more sophisticated manufacturing and service sector activities. These are: Bangladesh, Egypt, Guatemala, Kenya and Vietnam. Nine ICAs have been prepared for these countries, one in Vietnam and two in each of the other countries. A short mission was undertaken to each country to meet with government officials, private sector representatives and Bank staff. For each of the five countries, a case study was prepared. For the four countries with two reports, it was decided to combine them in a single presentation to avoid repetition and to enable the reader to better comprehend the evolution of the context and the process between the two products. Each of the nine ICAs was evaluated on four criteria: results; relevance; quality; 2 and dialogue (including dissemination). In each category, a rating (using the IEG 6-point scale) was given and an overall rating was derived from the category ratings. 1.6 This overall report is a synthesis of the five case studies. It highlights common features – as well as significant differences – among the ICAs and cross-cutting issues that apply to the entire sample. This report also serves as a platform for illustrating the findings and issues that emerged from this review. Approach 1.7 IEG reviews the Implementation Completion Reports (ICRs) of all World Bank projects and validates their ratings. The Bank spends a very similar amount of resources on Analytic and Advisory Activities (AAA) as it does on lending. But, in contrast to lending, there is no systematic review of AAA products by IEG to assess whether the institution and its borrowers were well served by the resources being spent on these products.3 1.8 But the review of analytical work differs significantly from IEG evaluations of Bank operations. First, AAA products do not normally include a statement of results or observable indicators. IEG needs to infer and interpret the results being sought, based on documents available and discussions with task teams and network. Second, there are no implementation completion reports (ICRs) which provide a basis for most of IEG‘s evaluations. The Activity Completion Summary (ACS) seldom provides the evaluative analysis found in ICRs. 1.9 The criteria used for rating are as follows:  Results: Were the objectives of the ICAs defined up-front and were they measurable and measured? Did the government follow up on the Bank‘s recommendations? Was there follow up through Bank analytic work and lending operations? Did the recommendations impact donor support?  Relevance is assessed in a number of dimensions. The first is the relevance of the topic as a development constraint. A useful test is to examine the role of investment climate in the Bank‘s assistance strategy. A second dimension concerns the ownership of the analysis and recommendations by the authorities. A consistency check with the national development strategy is often a good start. A third element concerns the timeliness of the exercise.  Quality: Were the ICAs of good technical quality? Were the surveys well prepared and implemented? Were they based on good analysis of the data? Did the second ICAs in the series provide inter-temporal comparisons with the results of the first ICAs? Did they do an effective job of benchmarking the country against appropriate regional and international comparators? Did they define recommendations in a clear and actionable way?  Dialogue and Dissemination: Were the reports the subject of a dialogue within the country? Were they disseminated widely to the government, civil society, and the donors? Were they packaged so as to address multiple audiences effectively? Was there systematic follow up in other Bank analytic work and policy dialogue? 3 IEG does, however, review AAAs in its Country Assistance Evaluations and Sector Evaluations. 3 1.10 The interpretation of the ratings under each criterion is shown in Table 1: Table 1: Rating Criteria Strategic Relevance and Dialogue and Rating Results Quality Ownership Dissemination HS Meets to a high extent both of Meets to a high extent all of Meets to a high extent all Meets to a high extent all of the following criteria: the following three criteria: of the following six criteria the following three criteria:  Addresses a key  Use of appropriate  Evidence of appropriate  Impact on government development constraint knowledge and analytic dissemination programs and/or the broader and is coherent with the techniques.  Report reaches effectively development dialogue in the country assistance program  Analysis of existing the right audiences through country  Delivered at the right time and/or new local data appropriate targeted  Impact on the design of the in relation to key decisions  Effective use of cross- distribution and events. Bank‘s program and/or the  Evidence of strong interest country comparisons  Evidence of sustained subsequent CAS by government, and global experiencea engagement development partners, or  Evidence of clear civil society and/or understanding of local evidence of active institutions and context. engagement of government  Clear and actionable agencies or local recommendations. institutions in conducting  Adequate review and the work client feedback S  Substantial impact on both of  No more than minor  No more than minor  No more than minor the criteria or substantial shortcomings in any of the shortcomings in any of shortcomings in any of the impact on one and high impact three criteria the six criteria. above areas. on the other MS  Substantial or higher impact  Moderate shortcomings in  Moderate shortcomings  Moderate shortcomings in on one criterion, modest or no no more than 2 criteria and in no more than 2 no more than two of the impact on the other no more than minor criteria and no more above criteria and no more shortcomings in the third than minor than minor in the third OR OR major shortcomings in shortcomings on the major shortcomings in one one criterion but no more remainder OR major criterion but no more than than minor shortcomings shortcomings in one but minor shortcomings in the in the other two no more than minor other two shortcomings in others MU  At least modest impact on  Moderate shortcomings in  Moderate shortcomings  Moderate shortcomings in one criterion, modest or no all 3 criteria or major in half or more of the all 3 criteria or major impact on the other shortcomings in 1-2 criteria criteria with no more shortcomings in 1-2 and no worse than moderate than minor ones in the criteria and no worse than shortcomings in the others OR major moderate shortcomings in remaining criteria shortcomings in fewer the remaining criteria. than half of the criteria with no more than moderate shortcomings in the others U  No impact on either criterion  Major shortcomings in all  Major shortcomings in  Major shortcomings in all three criteria majority of above three criteria. criteria HU  Negative impact on one or  Severe shortcomings 2 or  Severe shortcomings in  Severe shortcomings in 2 both of the criteria more of the criteria most of the criteria or more of the criteria Note: HS – Highly Satisfactory; S – Satisfactory; MS – Moderately Satisfactory; MU – Moderately Unsatisfactory; U – Unsatisfactory; and HU – Highly Unsatisfactory a. As appropriate, contingent on the nature of the analytic work. 5 2. ICAs as an Instrument of the World Bank Group 2.1 Since the early 2000s, the theme of investment climate has permeated virtually all operational products of the WBG, including AAA, investment lending and development policy loans. Diagnostic analysis of the investment climate is carried out through a number of specially designed exercises including ―Doing Business‖ – a global benchmarking of the regulatory framework, Investment Climate Assessments, Enterprise Surveys, and more specialized instruments for assessing corporate governance or measuring compliance costs. In the area of lending, the WBG has supported reforms of the investment climate around the world through investment loans, technical assistance, guarantees and development policy loans. In addition the investment climate has been an important focus of IFC advisory services. In 2002, the status of investment climate was elevated further, becoming one of the two pillars of the corporate strategy. In addition, it was the subject of the 2005 World Development Report: ―A Better Investment Climate for Everyone‖. 2.2 A major impetus for the rise in the profile of the investment climate came from the 2002 Private Sector Development Strategy. Following the collapse of centrally planned economies in the 1990s, the divestiture of state-owned firms became a key platform for Bank operations. Under the 2002 strategy, the emphasis shifted to the creation of a business- friendly environment for all kinds of enterprises. A raft of diagnostic products and lending instruments sprang up to meet the need soon thereafter. 2.3 At the heart of an ICA is an enterprise survey, a standardized firm-level survey that is used across regions.4 The survey includes both objective measures of the business environment and perception questions on the most relevant issues or constraints. As a standardized diagnostic tool, it facilitates cross-country comparisons by using a consistent definition of a core set of private enterprises that are included in the survey and by limiting the number of region and country specific questions. Structural differences across countries and regions are captured through additional sets of questions (called non-core modules). 2.4 The role of the ICA.5 A key objective of ICAs is to identify binding constraints on private investment and growth in the hope of motivating the country policy-makers to undertake reforms. ICAs are also expected to serve business units of the WBG by informing the formulation of country strategies and policy advice. The ICA offers a consistent way of measuring and comparing investment climate conditions across countries, replacing a number of divergent methodologies. It gives a snapshot of the country's business environment through the lens of private enterprises. 4 According to FPD staff, some of the surveys are conducted by Regional or Network teams with somewhat different methodologies. 5 The role of ICA as presented here is based on ―WBG PSD Strategy Mid-Cycle Implementation Progress Paper‖, June 2009, pp 14. 6 2.5 Accordingly, each ICA is expected to:  Look in detail at factors constraining efficient operations and firm productivity6, including product markets, infrastructure and regulatory and institutional framework;  Provide tools and analytical framework to help governments develop policies and programs that support productivity growth;  Provide the private sector with facts to support dialogue with government;  Track changes in the investment climate over time and across countries. 2.6 An ICA typically covers a variety of broad topics, including the legal framework, infrastructure and finance. While there is a similarity in the cluster of key issues within each Region, there are variations and differences in emphasis. In East Asia, for example, ICAs devote considerable attention to property rights, corporate restructuring and financial sector reforms. In Africa, the role of public enterprises and land tenure features prominently. 2.7 A related and perhaps better known knowledge product of the FPD network is ―Doing Business‖ (DB), which focuses on business regulations and other factors affecting the cost of doing business from a firm‘s perspective. DB makes use of assessments by experts (principally lawyers), rather than firm-level surveys, to measure different aspects of compliance costs. As discussed in a recent IEG evaluation, DB has contributed to development by providing countries with a basis for international comparisons of their regulatory regimes.7 It has helped to catalyze debates on investment climate in developing countries. DB, however, has some methodological limitations. For example, it does not measure costs and benefits of regulations from the perspective of society as a whole. 2.8 There are significant differences in the contents of ICAs and DB reports. First, DB indicators are based on the assumption that de jure regulations are effectively applied. To the extent that implementation differs, both in extent and in kind, from the intent of regulations, actual business experiences may not correspond to those envisaged in DB. ICAs, on the other hand, are based on direct observations at individual firms. They reflect actual (de facto) measurements or perceptions of businesses. Second, DB indicators allow for comparisons across countries but not across sectors or regions within the same country. ICAs and enterprise surveys, which are more costly to implement, can provide either national-level estimates or aggregated indicators at industry, regional or firm levels. 6 ICAs generally provide measures of labor and total factor productivity (TFP). Labor productivity is the value- added produced by each worker, or mathematically the value-added divided by the number of workers. Total factor productivity, a theoretical construct, is defined as the residual of output or value-added that cannot be explained by changes in labor and capital. Changes in the TFP are often attributed to the investment climate. The estimates of TFP are based on technical considerations, including production function and regression analysis, which are beyond the scope of this report. 7 IEG (2008). Doing Business: An Independent Evaluation, The World Bank. 7 Figure 1. Number of ICAs Delivered, FY03-FY11 25 22 20 20 18 17 15 15 11 10 7 7 5 2 0 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 Notes: FY11 data is a preliminary estimate. Source: Business Warehouse. 2.9 Since 2002 when the ICA was introduced and piloted, the WBG has produced more than 100 reports covering more than 90 countries (half of which are IDA eligible) across all Regions. In more than twenty countries, two or more ICAs – including updates – have been written. The interval between ICAs ranges from 3 to 6 years, with a median of 4 years. Most – but not all – of the reports address issues at the national level. Some of the ICAs deal with regional units, like the FY10 Regional ICA for the Southern African Development Community (unpublished). Some ICAs take a sub-national or sector perspective, including the FY05 ICA for Orissa (a state of India) and FY07 Rural ICA for Ethiopia.8 2.10 Division of responsibility. The preparation of ICAs generally involves close collaboration between the World Bank, IFC and MIGA, with the responsibility for ICA spread across many Networks and business units. In most cases, the FPD sector units in the Regions take the lead in initiating and writing ICA reports. While this review is underway (during FY2011), these sector units are generally nested within PREM, except in Africa and ECA where they belong to a self-standing FPD Department outside of the PREM Network. In four out of six Regions, the FPD units report to a PREM Director. 2.11 In most cases, the enterprise surveys are managed centrally by the Enterprise Analysis Unit, which is also part of the FPD Vice Presidency. With its own funding, the Unit initiates and manages the surveys across Regions on a rolling basis, aiming to achieve full coverage every three years. Country Units also have the option of initiating the survey themselves. The fieldwork is conducted by specialized firms with the required technical expertise, which can be international or local. Where the capacity exists for carrying out surveys of this kind, but there is limited on the ground experience, FPD has provided 8 The case of Ethiopia is among the pilots for a new line of product: Rural Investment Climate Assessment (RICA). 8 technical support to local consulting firms, institutes or think-tanks to carry out the survey. In principle, quality assurance is the responsibility of the FPD Sector Board, which is under the joint World Bank-IFC Vice Presidency for Financial and Private Sector Development. A panel of ICA experts has been set up by the Board to provide technical support for task teams and to give clearance to completed work. A standard practice is to rely on one of the experts on the panel to serve as peer reviewer. In some cases, members of the expert panel may also join the task team and take up production responsibility. IEG has been informed that the structure of the FPD Vice Presidency, including the units in the Regions, is currently (March 2011) being redesigned. It is expected that all Regions would soon have a self standing FPD Department, like those of Africa and ECA today. 2.12 Independent review. In 2006, IEG conducted a review of lending and non-lending services under the rubric of investment climate for the period 1992-2003.9 The evaluation found that the survey-based diagnostics contributed to development research by creating benchmark indicators that are comparable across countries. ICAs were also found helpful in the formulation of growth-oriented reform agenda and in motivating many countries to undertake important reform measures. However, there was often an excessive focus on the manufacturing sector during that period while institutional analysis was often not adequate to support the design and sequencing of politically sensitive reform measures. 2.13 The 2006 evaluation also noted a few organizational issues, some of which remained in place during the period covered by the current review. One issue is the division of labor across sector units within some of the Regions. The competition between FPD and PREM for leadership on the reform agenda may have abated in some cases, but remained intense in at least one of the ICAs covered in this PAR (Vietnam) and resulted in a breakdown in communication. Another issue is the analysis of institutions and political economy of reforms.10 For the ICAs reviewed in this PAR, the finding of the previous IEG review on this issue remains largely valid today.11 2.14 ICA Budgets. The cost of an ICA varies, depending on the scope of coverage and size of the country. At the high end, the cost for India, Russia, and Pakistan, ranged from $700,000-900,000.12 Considerably less expensive were the ICA for Zanzibar at $25,000 and for Benin at $28,000. For the ICAs reviewed for this PAR, a similar pattern holds, as shown in the Table 2 below. 13 The cost ranged from less than $50,000 to more than $500,000. In addition, the change in cost can be substantial over a period of just a few years. The case of Kenya is typical, with the price tag more than tripling just 3 years. The variability is much smaller with respect to delivery time, as shown in Table 3 below. Still, an ICA can take as little as 10 months or as long as 29 months while the Guidelines stipulate an expected delivery time of 8 months. 9 IEG (2006), Improving Investment Climates – An evaluation of World Bank Group Assistance. 10 As discussed below, institutional and political analysis have important implications for ICA effectiveness. 11 In response to IEG recommendations, FPD management said that it was ―cognizant of the importance of analytical work on institutional arrangements and the political economy of reform‖ and that it was preparing (i) guidance notes; (ii) assessments of constraints and incentives; and (iii) 15 country case studies. See IEG (2006) 12 According to data from WB Business Warehouse and Operations Portal. 13 These figures pertain to the cost of preparing ICAs not including the enterprise surveys. 9 2.15 For Bank management, the information available seems to be of little help for planning and decision making. Faced with the request for an investment climate assessment, Bank managers can hardly find a good benchmark for preparing a budget or projecting a delivery date. Perhaps this is already widely recognized among the managers involved in the sample countries, as the task budgets did not appear to serve as an instrument of financial control.14 In most cases, actual expenses exceed budget estimates by a large margin. For example, the cost of the 2008 Guatemala report was 260 percent of the budget, while that of the 2008 Bangladesh report was 1094 percent. Table 2: Planned and Actual Costs of Selected Investment Climate Assessments Country Delivery Date Project ID Budget ($K) Actual ($K) Bangladesh 03 FY03 P077465 50 161 Bangladesh 08 FY08 P099838 50 558 Egypt 04 FY06 P089111 350 334 Egypt 09 FY09 P108292 175 301 Guatemala 04 FY05 P083167 40 49 Guatemala 08 FY08 P104173 51 134 Kenya 04 FY05 P080307 250 135 Kenya 08 FY08 P102213 30 419 Vietnam FY07 P088218 10 196 Source: WB Operations Portal Note : In the case of the Egypt. FY09 ICA, the figure for the original budget does not include $100,000 of support promised up front by the FPD anchor, but not provided and therefore funded as part of the overrun. Table 3. Time for Completion - Selected Investment Climate Assessments Country Project ID Start Delivery Duration(M) Bangladesh 03 P077465 6/3/2002 4/7/2003 10.3 Bangladesh 08 P099838 3/15/2006 5/30/2008 26.9 Egypt 04 P089111 9/24/2004 8/1/2005 10.4 Egypt 09 P108292 2/27/2008 6/29/2009 16.3 Guatemala 04 P083167 5/8/2003 8/13/2004 15.4 Guatemala 08 P104173 1/18/2007 6/26/2008 17.5 Kenya 04 P080307 7/1/2002 11/30/2004 29.4 Kenya 08 P102213 7/27/2007 6/16/2008 10.8 Vietnam P088218 7/30/2004 1/3/2007 29.6 Source: WB Operations Portal 14 This review focuses on the results (outcomes) of the selected ICAs and does not directly evaluate the inputs, including the staff time and resources, devoted to the preparation of the reports. 11 3. The ICA Experience in the Five Countries 3.1 As indicated above, the core of each ICA is an enterprise survey with a typical size of 500 to 1,500 firms or at a minimum 200.15 Initially the surveys were limited to manufacturing enterprises with more than 10 workers located in one or two of the main cities. At the request of Governments, in many countries the survey has been broadened to include services, smaller and informal enterprises with as few as two or three workers, and enterprises located in smaller towns and even rural areas. Today, under a global methodology, the coverage is the entire non-agricultural private economy, including all manufacturing and service firms with a least 5 employees in the main geographical areas. 3.2 The surveys generally focus on a set of constraints drawn from the Bank‘s sector analysis.16 These constraints are generally discussed in advance with the core ministries and with the private sector to ensure that the key areas are captured.17 In addition the surveys ask enterprises for key financial data on costs, sales, investments and how they have changed over time. The top four or five constraints identified by the enterprise surveys underpinning the ICAs reviewed in this PAR are presented in the Table below: Table 4. The Core Investment Climate Constraints Identified In ICAs Energy & Report Inflation Governance Transport Finance Land Skills Taxation Security Licensing Bang 03 X X X X Bang 08 X X X X X X Egypt 04 X X X X X Egypt 09 X X X X X X Guat 04 X X X X X X Guat 08 X X X X X X Kenya 04 X X X X X Kenya 08 X X X X Vietnam X X X X Source: IEG country specific case studies prepared for this report (See Annexes). 3.3 Bangladesh: Two ICAs have been produced for Bangladesh. The first, prepared in 2001/2 and published in 2003, was one of the first of its kind and had a significant input from the central unit in developing the survey and the basic approach. A major launch event was 15 The survey sample frame is generally derived from the population of eligible firms obtained from the country‘s statistical office. Sometimes, however, it is necessary to obtain the master list of firms from other government agencies such as tax or business licensing authorities. In some cases, the master list is obtained from business associations or marketing databases. In a few cases, the sample frame is created via ―block enumeration‖, where the World Bank ―manually‖ constructs a list of eligible firms after 1) partitioning major cities into clusters and blocks, 2) randomly selecting a subset of blocks which will then be enumerated. 16 Common issues are set out in the ―core questionnaire‖ which is used across countries. 17 To capture country specific issues, supplemental modules of questions are added to the core questionnaire. 12 staged with participation from senior Washington managers and high-profile press coverage. The message that Bangladesh was losing two percentage points of GDP growth each year because of inadequacies and interruptions in power supply was especially resonant. 3.4 The focus of the first ICA was essentially diagnostic with the policy recommendations implicit in the identification of four key issues – infrastructure, particularly power; access to finance particularly for small and medium enterprises, bureaucracy and red tape, and corruption. The ICA, together with the ―Doing Business‖ surveys, moved the discussion of investment climate issues to center stage, contributing to a decision by DFID and IFC to establish a Bangladesh Investment Climate Facility (BICF) in 2006. It was meant to fund studies and technical assistance to support Government efforts in improving the investment climate particularly with respect to the regulatory environment and bureaucratic interventions, with the expectation that this would also lead to better governance. 3.5 At the beginning of 2007, with the domestic political situation becoming increasingly chaotic, the Bangladesh military intervened and installed a caretaker government headed by the then Governor of the Central Bank, Dr. Fakhruddin Ali Ahmed. The BICF proposed to the new authorities that two new public/private institutions be set up to address investment climate issues: A Regulatory Reform Commission (RRC) and a Better Business Forum (BBF). Following Government appointment, these two agencies proceeded rapidly to simplify and streamline regulation. 3.6 In the course of 2006/7, the WBG embarked on a follow-up ICA. The previous survey had been limited to the cities of Dhaka and Chittagong, and had only covered manufacturing enterprises with more than ten workers. This time around it was decided to broaden spatial coverage to include peri-urban and rural areas in a large number of localities. Since most rural firms had less than 10 workers, this decision also required enlarging the population of micro enterprises. In addition, since services and construction accounted for a major share of rural enterprises, they were to be covered as well. The new ICA was published in 2008. It confirmed that the issues of the previous ICA remained relevant. Furthermore, two important new issues emerged. The first was the issue of access to land as an increasingly important constraint; and the second was the availability of skilled labor in various sectors. The BBF formed working groups to study these issues further and at the request of the country director, the ICA team went beyond the diagnostics and made a set of policy recommendations for the Government and donors to follow up. 3.7 The 2008 elections produced a resounding victory for the Awami League. The new Government rejected many of the initiatives that had been undertaken by the caretaker government – on political rather than substantive grounds. Among the casualties were the BBF and RRC, despite recognition of the useful role that these two agencies were playing. The reform agenda was left to individual ministries to follow up, and although there was some incremental progress, most of the bolder measures that had been proposed were off the table. BICF continues to exist and has shifted its focus to much-needed capacity building efforts while the Bank continues to take up investment climate issues through its PRSCs which have considerable influence in Bangladesh. 13 Box 1. Bangladesh: The Power and Limitations of ICAs The ICA experience in Bangladesh illustrates some of its strengths and weaknesses, which are also observed in the other countries covered in this review. A key motivation for conducting the 2003 Bangladesh ICA was to provide concrete evidence for the problems in the supply of electricity. The issue had long inhibited investment and growth in the economy. Results of the enterprise survey painted a disturbing picture. The poor quality and reliability of power supply forced most private firms to incur substantial costs of running in-house private generators. In addition, the ICA analysis showed that inadequate power supply depressed economic growth by about two percentage points each year. The findings resonated deeply with the general public and the incumbent administration. While powerful, the evidence was not enough to overcome deep-seated governance issues. The ICA helped the WBG advance its country dialogue and encouraged the Government to initiate regulatory reforms and preparation for new investments in the power sector. But ultimately little was achieved in terms of new power generation or expanded distribution networks. Corruption and political feuds prevented large-scale power projects from being implemented. 3.8 Egypt: In 2004, Dr. Mahmoud Mohieldin was appointed as the head of a newly created Ministry of Investment (MOI) designed to oversee the privatization program and promote the development of new investment, including the establishment of Special Economic Zones. While initially lukewarm to the proposed ICA, Dr. Mohieldin and his colleagues in the Ministry soon saw the potential of the instrument as a tool for benchmarking the impact of the measures that he and his colleague, Dr. Youssef Boutros Ghali (a former IMF staff member who became Minister of Finance in 2004), were proposing. The Ministry threw its weight behind the survey and asked the business community for cooperation. 3.9 Three surveys and assessments have since been undertaken – in 2004 (finalized in 2005), 2006 (an update with a report produced in 2007) and 2009. This is an unusually high frequency and reflects the fact that these reform-minded Ministers found the ICA of value – both to set the reform agenda and, perhaps more importantly, as an independent assessment of progress in implementation of the reform. The striking improvement in perceptions of the business community on taxation that the ICA showed helped to maintain the support of the Prime Minister for continuing the reform and to build a consensus within the Government. The MOI became the principal audience of the ICA. It reviewed and discussed the document carefully, but – in line with Government policy on documents dealing with sensitive issues such as the prevalence of corruption – the ICAs were not made public. Dissemination among the business community and academia was limited to PowerPoint presentations. 3.10 The initial ICAs were confined to enterprises with 10 employees or more in Cairo and Alexandria. For the 2009 ICA, Dr. Mohieldin asked that the ICA include coverage of smaller and informal enterprises, and particularly of enterprises in Upper Egypt where per capita incomes were lower and poverty rates higher than in the rest of the country. The ICA survey did not come up with any major surprises. But this finding in itself was useful in confirming that there was no magic bullet to be found in promoting Upper Egypt. While access to land proved less of an issue than in Lower Egypt, all forms of infrastructure and 14 particularly costs of importing and exporting loomed large as issues in Upper Egypt. The survey also indicated that, while access to finance was not a problem for large enterprises, it was a serious constraint for smaller ones which relied exclusively on their own resources for investment. 3.11 Guatemala: After signing the Peace Accords in 1996, the government of Alvaro Arzú started reforms aimed at accelerating growth and reducing inequality. Increasing productivity and improving competitiveness was one of the tools to raise growth. The Bank sought to help by granting a loan for competitiveness ($30 million) in 2000 – the last year of the Arzú government. The loan envisaged doing a survey of Guatemalan competitiveness, and the Bank‘s FPD staff in the region used the survey as an opportunity to do an ICA report. The administration that followed (Alfonso Portillo) showed no interest in the loan and its execution stalled. 3.12 In early 2003, the five nations of Central America (Costa Rica, El Salvador, Guatemala, Honduras, and Nicaragua) embarked on negotiations for a Free Trade Agreement with the United States. The negotiations made Guatemala realize that it had to improve its investment climate to benefit from CAFTA. The proposed report was seen as an appropriate tool to diagnose the problems. The report would focus on information at the firm level in four areas: (i) regulatory environment; (ii) access to and costs of credit; (iii) quality of infrastructure; (iv) innovation, technology and human capital formation. The expectation was that the analysis would help identify the essential reforms for private sector development, foreign investment and sustainable growth. The report was completed one year behind schedule and was published in December 2004. By then, the Berger administration had taken over. For its analysis, the report used the results of a survey on the investment climate in the urban manufacturing sector. The Bank produced a report of high strategic relevance whose results the government owned and used to raise awareness of the problems, help it define an agenda for competitiveness and serve as baseline values when applicable. During 2004-07, the government carried out reforms that led to an improvement in the investment climate, as documented by the ICA 2008 ICA. 3.13 The government of Oscar Berger (January 14, 2004-January 14, 2008) prepared a competitiveness agenda for 2005-2015. The agenda had six strategic axes, some of which dealt with aspects identified by the investment climate assessments. As indicated, the government had used the ICA 2004 to raise awareness of the issues and to establish a baseline where progress in competitiveness could be measured. At the Government‘s request, the Bank carried out the enterprise surveys and presented its preliminary findings to the government in July 2007. A draft of the report was discussed in February 2008 and the final report completed in June 2008. The report shows an improving investment climate during 2003-2007. One contrary indicator is with regard to energy, where the perception points to declining quality, although the objective evidence indicates improvement. 3.14 Kenya: Years of neglect had left the country's infrastructure in severe disrepair, including a massive backlog in road maintenance, the collapse of railway services – which reduced its share in transportation from 60 percent in 1990 to 6 percent today – and the near collapse of airport services (except for cargo). The number of air travelers rose from 2.1 million to 5.1 million with constant airport capacity. Corruption has been perceived 15 throughout the past two decades as widespread, often preventing official financing of projects meant to address development needs. The prospect for the private sector improved, however, with the inauguration in 2002 of the National Rainbow Coalition, which showed an interest in the future of private businesses. There was an increasing recognition that the private sector would drive faster economic growth while reducing unemployment and poverty levels. The 2004 ICA picked up on Kenya‘s renewed interest in the private sector. Soon after the Bank initiated the ICA, Government started preparing its own Private Sector Development Strategy. 3.15 Despite the renewed interest in private sector development, little progress was achieved. Implementation of the Government‘s Economic Recovery Strategy for Wealth and Employment Creation (2003-2007) did little to put in place specific interventions to spur private investment. In January 2007, the Government issued a Private Sector Development Strategy 2006-2010, which aimed to ―enhance growth and competitiveness‖ and ―catalyze the provision of an enabling environment.‖ Following the release of the Strategy, the Bank embarked on the preparation of the 2007 ICA survey. It was broader than the 2003 survey in that it covered small businesses as well as the service sector. The results of the two surveys provided an opportunity to analyze time trends in manufacturing variables. However, the potential impact of the ICA was diminished by political developments. Kenya suffered from a contested election in December 2007 and widespread ethnic violence in early 2008. GDP growth moderated from 7.1 percent in 2007 to 1.7 percent in 2008. Although the disruption calmed down as a new coalition government was put in place under international pressure, exogenous and institutional factors led to only patchy progress. While a new Constitution adopted in 2010 strengthened the anti-corruption law, real and sustained change still remains to be seen. 3.16 Vietnam: The case of Vietnam differs from the others reviewed in this PAR in that it has only one report, the 2006 ICA. The Bank‘s initiative for the ICA took place at a time when the Vietnamese economy was rapidly evolving from a state-directed model towards a market economy – when there was healthy momentum to address obstacles in the investment climate. The ICA‘s main focus on the question of whether there was a level playing field between state-owned and private firms seemed highly relevant at the time. At the same time, the Unified Enterprise Law and Investment Law were also being drafted. The Bank, including ICA team members, was involved as an advisor on these topics to the Prime Minister‘s Consultative Group. The survey findings served as one of the inputs, but the main impetus for reforms came from Vietnam‘s campaign for accession to the WTO. The two laws represented major steps towards leveling the playing field between private and state- owned companies. 3.17 At the time when the first draft report was prepared, the original TTL handed the report over to her successor, which caused a delay to an already tardy product. This meant that the ICA, with its focus on leveling the playing field and a final report that was quite delayed and unpublished, had lost its ability to impact the reform agenda. Instead, the findings of ICA were absorbed into the 2006 Vietnam Development Report (VDR) – an annual multi-donor effort presented at the time of the Consultative Group meeting. 17 4. Results of the ICAs 4.1 On the one hand, ICAs are expected to help the WBG advance the country dialogue and to provide internationally comparable micro-economic data. ICA reports serve management by providing new knowledge for strategy formation, project preparation and donor coordination. On the other hand, broader objectives are envisaged with respect to client countries, including, (a) influencing government policy, (b) building local capacity and (c) stimulating public debate. By and large, ICAs as a family of analytic tools have performed these functions. According to an IEG study, ICAs are among the most frequently cited knowledge products by Bank staff and in-country stakeholders, along with core Bank reports such as CEMs, PERs and FSAPs.18 4.2 On closer examination, the results of each ICA can be evaluated at two levels. The first relates to the diagnostic: Did the ICA give an accurate assessment of the constraints to investment and their relative importance? Did it provide an evidence base for ‗what everyone knows‘ and take off the table the doubts that might be voiced about particular issues or the suspicion that this is just the World Bank pursuing a particular agenda? The second level of results is the impact on the ground, i.e., the measures the Government took to address the concerns that were raised in the ICA. 4.3 This evaluation focuses on this second level in addressing results – impact on the ground – while acknowledging the value of the evidence base that the ICAs represent. In practice all the ICAs reviewed went well beyond this and discussed what the data was saying about the nature of private sector development in the country, the relative priorities of the different constraints identified, and the steps that needed to be taken to address these constraints. The issue for evaluation is therefore how well the ICAs articulated these steps and the follow-up actions that were taken in response. 4.4 The results of the ICA are very much influenced by the political receptivity of the Government of the day. In both Bangladesh and Guatemala the first ICAs in the series were undertaken or published at a time when there were Governments in place that saw the potential for using the agenda identified in the ICA as a basis for policy reforms and benchmarking achievements against future surveys. By the time the second ICAs took place in these countries Governments had changed and the enthusiasm of the previous Government tended, if anything, to work against the interests of the key reforms that had been identified. In Kenya the situation was reversed, with a disinterested Government at the time of the first ICA giving way to a much more responsive Government when the second ICA was released. 4.5 The results of the ICA also reflect the extent to which the diagnostics and core recommendations were embedded in the Bank’s subsequent lending and analytic work. In Egypt the Bank used the ICA as the centerpiece of its dialogue with the Minister of Investments and carried the ICA analysis into follow-up analytic work and development policy loans. 18 IEG (2008). Using knowledge to improve development effectiveness: An evaluation of World Bank economic and sector work. 18 4.6 A third element of the ICA results is the effects on public opinion. This is reflected in the degree to which the diagnostics helped to develop a common understanding of the constraints to private sector investment and to maintain the general direction of change even if the specific follow up measures were held hostage to political expediency. In two cases, Egypt and Vietnam, the ICAs were only discussed in full with the central government although PowerPoint presentations were made to the business community that participated in the survey and there was some press coverage. By and large, the unavailability of the reports has meant that the ICAs have had a limited effect on public awareness and debate. Box 2. Guatemala: The Tale of Two ICAs They were of comparable merits in terms of relevance and technical quality. And yet the results were dramatically different. The first ICA (2004) was influential and much appreciated while the second (2008) was left to the dustbin of history. As discussed in Chapter 3, there was a change of Government in between. The Berger administration (2004-08) was highly motivated to improve the investment climate as Guatemala began to implement CAFTA – the free trade agreement between the US and central American countries. The first ICA was thus in the right place at the right time and had a large impact. The second ICA arrived in 2008, when the Colom administration was in charge. Investment climate was no longer a top priority. It seemed like a chance event; the second ICA simply had the bad luck of finding an unreceptive counterpart. But luck was just part of the story. Good planning would have made a difference. In the 2007 elections, policy agendas of the leading parties were known well in advance. The country unit and ICA team could have evaluated alternative outcomes and taken actions to mitigate the risk. In fact, by January 2007, when the ICA was initiated, candidate Alvaro Colom of the National Unity of Hope already had solid lead (15 percent). The fate of the second ICA was sealed before it was written. The results of ICAs in the five countries 4.7 Bangladesh ICAs. Results of the first report (2004) are rated Moderately Satisfactory. It introduced the enterprise sampling and survey instrument to Bangladesh and helped build capacity to carry out such surveys. It also provided useful validation of ―what everyone knew‖, especially regarding the issue of power shortages as a key constraint. Together with other analytical work of the Bank, such as Doing Business Indicators, the ICA helped move the discussion of investment climate issues to center stage, and thereby contributing to the decision by DFID and IFC to establish a Bangladesh Investment Climate Facility (BICF) in 2006 to support the Government on this agenda. The second ICA (2009), however, proved less fortunate. The timing of the report coincided with the arrival of a new government that was less responsive to the concerns of the private sector. The reform momentum dissipated with no further significant measures implemented. At this point, the Bank‘s efforts on IC were mostly channeled through the BICF or through lending as with the Development Support Credits (DSCs). Follow-up activities were less extensive than that of the first report. The results of the 2009 report are rated Moderately Unsatisfactory. 4.8 Egypt ICAs. Results of the 2004 report are rated Highly Satisfactory. The timing of the first ICA was ideal in that Dr. Mohieldin, newly appointed as Minister of Investments, was looking for an instrument to bring other ministries into the reform agenda. After 19 questioning the ICA approach initially he quickly saw the potential role that the instrument could play. The ICA series was also valued as an instrument for benchmarking the progress that the Ministry was achieving and therefore helped to prioritize the Ministry‘s efforts. The Government found the information on the frequency of inspections of private enterprises by various government agencies, particularly useful. This enabled the Ministry of Investments to take a stand on reducing these inspection visits which in their view simply produced time- wasting and led to informal payments. The analysis of the problems of Upper Egypt contributed to proposals to enhance the infrastructure in that area. One area with modest progress was access to land where the interests of the military prevented policy action. 4.9 On the Bank side, the work that was undertaken contributed to the financial sector work as well as the 4th DPL. Arguably the DPL and the decision to proceed with policy lending grew out of the ICA. In the words of one staff member, the ICA gave the Bank a foothold on policies in Egypt, where it had had none before. The seriousness with which the Government took the ICA can be seen from the request that in the next ICA a component be included covering issues of Corporate Social Responsibility. 4.10 Results of the 2009 ICA are rated Satisfactory. The rating reflects the perception in the Ministry of Investments that in the areas over which they had direct control, the measures that needed to be taken were already in place. In areas which depended on action by other Ministries (other than Finance which was fully on board with the reform agenda), progress was slower. Although the ICA helped in the internal dialogue, because it was not made publicly available, it could not be used to as an instrument for influencing public opinions. 4.11 Guatemala ICAs. Results of the 2004 ICA are Satisfactory, but those of the 2008 report Moderately Unsatisfactory. The two are very similar in their approach and contents. But the results achieved are sharply divergent. 4.12 In 2004, the negotiations of the Central America Free Trade Agreement (CAFTA) – which included the U.S. and Guatemala along with four other Central American countries – generated strong interest in the investment. The Berger administration moved expeditiously to upgrade Guatemala‘s competitiveness in order to gain a foothold in international markets. A new competitiveness agenda was declared, with legislation passed (including a new law governing public-private partnerships or PPP), an agency was created and new projects were approved by the Bank. The ICA report was widely credited for motivating the reforms and providing the analysis and recommendations. Of particular value was discussion of Guatemala‘s position relative to the rest of the region. Within the Bank, the 2005 CAS, 2007 CAS Progress Report and CEM (2004) referred to the findings of the ICA. 4.13 The strategic direction of the Government changed dramatically with the arrival of the Colom administration (January 2008). The Government was no longer willing to make concerted efforts or bear the political costs of pursuing the competitiveness agenda. No new reform measures recommended by the ICA were implemented. The report was delivered to the Ministry of Finance and discussed with key local partners. Bank support for private sector development continued but became less active, with one CEM and one MSME project approved. The CEM built on the 2008 ICA and focused on business issues faced by SMEs. But no new assistance strategy was presented during this administration. 20 4.14 Kenya ICAs. The results of both the 2004 and 2008 ICAs are rated Moderately Satisfactory. Both ICAs clearly set out the analytical objectives, which included an assessment of the performance of manufacturing firms, identification of the key constraints on growth and recommendations to promote the private sector. Concrete results have since been achieved, but few are unambiguously attributable to the ICAs. 4.15 Most respondents have seen improvements in power supply, the road network and access to credit. However, new electricity regulations – introduced in 2004 and 2006 – resulted mainly from the Bank‘s sustained dialogue in the power sector. On transport, the key measures (increase in road funds and the decentralization of authorities enacted in 2007) are included in the recommendations of the 2004 ICA. Moreover, much of improvement in credit access is related to better macroeconomic management and a robust growth of ―mobile money‖ made possible by an expansion of cell phones. 4.16 On corruption, the ICAs may have contributed to broader Bank dialogue which has led to many new anti-corruption laws. Nonetheless, the perception of corruption remains unchanged. There is clear evidence, however, on the use of the ICA findings by domestic private advocacy groups, including the Kenya Association of Manufacturers (KAM) and the Kenya Private Sector Alliance (KEPSA). 4.17 Within the World Bank Group, the ICAs have found a wider audience. The 2004 country assistance strategy (CAS) made the investment climate one of the four key areas for Bank support. The 2010 CAS drew on the 2008 ICA and proposed Bank support for regulatory reform, road and port rehabilitation and new investment in power and water, as recommended. In addition, ICA findings were used extensively in the Kenya Growth and Competitiveness Report (2005) and in the Kenya Accelerating and Sustaining Inclusive Growth (2008). 4.18 The Vietnam ICA. Unlike most of the ICAs under review, this report does not define the objectives or results framework within the report. Nonetheless, at an earlier stage (October, 2004), the Concept Note for the ICA clearly set out the diagnostic results to be achieved and the objective of informing the Government on the major legislative agenda, including the drafting of the Unified Enterprise Law and the Common Investment Law. 4.19 Most of the ICA work was conducted during the Government‘s campaign to join the WTO. A major part of the effort was to introduce the requisite legal and regulatory reforms. The ICA team provided policy briefs on investment climate issues to the Prime Minister‘s Research Advisory Group. In 2005, the Unified Enterprise Law and the Investment Law were enacted. The reform represented major progress on the key issues raised by the ICA – that of commercial disadvantages of private firms in favor of state-owned enterprises. 4.20 The ICA‘s contributions to this landmark legislation, however, were relatively small. For one thing, the report was never published or widely circulated. Only a few Bank staff and government officials saw the final draft of the report. For another, unexpected turnover of ICA task managers substantially delayed the completion of the final report. When it was finally delivered to the Government in November, 2007, almost two years after the passage 21 of the key legislation, the official response was to point out – diplomatically – that the findings were out of date. Overall assessment 4.21 During the period under review (which varies from one country to anther), the investment climate has improved in the five countries – significantly in some cases and less so in others. For the most part, the ICAs have made a contribution. Often, the contribution is substantial and unequivocal, as with the case of 2004 ICA in Egypt and 2003 ICA in Guatemala. In other cases, the contribution is less direct but still significant and conveyed through Bank dialogue and sector work, as with the case of Bangladesh 2004 ICA and the Kenya 2008 ICA. Some of the ICAs may have exerted an influence primarily through the work of local advocacy groups and through a heightened awareness of the underlying issues among policymakers and the general public. 4.22 ICA results are, however, highly sensitive to external factors beyond the control of the Bank, including especially the receptivity of the incumbent government. Within the same country, the results achieved can be dramatically amplified by strong government support or can quickly diminish following a change in incumbency, as indicated by the experience of Guatemala and Bangladesh. Furthermore, the degree of receptivity can span a wide range – from strong support in some cases to virtual disregard in others. The variability of attitude is more pronounced in the case of ICAs than other diagnostic work such as the CEM or PER and seems unique among the Bank‘s knowledge products. 4.23 Six out of nine ICAs reviewed are rated highly satisfactory, fully satisfactory or moderately (See Table 5). Only three are less than satisfactory and, in all of these three cases, negative external events played a significant role. These ratings, it should be noted, are not necessarily a reflection of Bank performance. Table 5. Results Ratings Report Results Ratings Bangladesh 2003 Moderately Satisfactory Bangladesh 2008 Moderately Unsatisfactory Egypt 2004 Highly Satisfactory Egypt 2009 Satisfactory Guatemala 2004 Satisfactory Guatemala 2008 Moderately Unsatisfactory Kenya 2004 Moderately Satisfactory Kenya 2008 Moderately Satisfactory Vietnam 2006 Unsatisfactory Source: IEG Note: These ratings are the evaluators‘ judgments regarding the outcomes of each ICA. They are not an assessment of the performance of the Bank, the task team or the Government. 23 5. Strategic Relevance of the ICAs 5.1 The relevance of the ICAs is normally assessed in terms a number of dimensions. The first is the relevance of the topic as a development constraint. Obviously the Investment Climate is a relevant development issue for most developing countries. This said the interest in assessing the quality of the investment climate is likely to be somewhat less in middle income countries with high levels of private investment and FDI and in very low income countries and fragile states where it may be more critical to analyze and deal with other constraints. It is in the transforming economies that the ICA issue becomes most relevant. The five country cases all fall into this category. This was a deliberate choice in order to be able to focus the study on the factors that determine the follow up on the ICA recommendations. A consideration here is the extent to which the topic is given priority in the Bank‘s assistance strategy and country dialogue. If the investment climate is considered the predominant constraint in the country or is one of the key pillars of Bank assistance, it is a clear evidence of the ICA‘s relevance – at least to the Bank if not to the client. 5.2 A second dimension of relevance concerns the ownership of the analysis and the recommendations. This is not ownership in the sense of the ICA being perceived as a Government product. On the contrary, much of the credibility of the ICA comes from its relative independence and the fact that the business community is comfortable in responding frankly to the questions of the interviewers. The ownership is rather a matter of the Government seeing the ICA as a tool for benchmarking progress on the investment climate – supplementing the Doing Business Indicators in this regard, and setting the agenda for policy reform. This is exactly the role the ICA played in Egypt, where the Minister of Investments skillfully used the ICA within the Government in order to bring other Ministries with key roles in investment policy reform into a consensus. 5.3 It is often argued that there is a need to distinguish here between whether a product is demand-driven and requested by the client or supply-driven. In the case of demand-driven studies the request itself is usually interpreted as evidence of client ownership. In the case of supply-driven work, the Bank identifies an issue that is strategic and significant, and uses its analysis as a way of generating interest. In general the ICAs have fitted this latter model. From an evaluation standpoint however, the two approaches raise very similar questions. It is often the case for example, that a demand-driven study reflects the interests of a particular minister or official and is not of great interest to his or her colleagues. When the ‗champion‘ moves or leaves office, the study may be left without ownership. It is therefore important to ensure that even demand-driven studies meet criteria for strategic relevance. For supply- driven ESW there is a need to agree with the client on the topic and scope, and bring the client into the discussion. Sometimes the Bank fails in this endeavor because it simply goes its own way to research a key issue in the conviction that the report will speak for itself. 5.4 A third issue concerns the timeliness of the ICA, i.e., was the report was produced at a time appropriate to affect development partner decisions. While this is an important issue for the evaluation of relevance for most products, it could be argued that it is less significant for the ICA given the fact that it is a periodic survey that is intended to be undertaken on a 3 to 5 year cycle. In this sense timeliness becomes more of an efficiency issue than a matter of relevance i.e. during periods when there is less interest in the topic, it may make sense to 24 prepare a short report at lower cost and undertake a much lighter dissemination process, than during periods when there is the potential for real traction from the ICA conclusions.19 In Bangladesh for example, an extremely costly and time-consuming analysis was undertaken and published at precisely the nadir of Government interest in the investment climate. 5.5 A fourth issue for relevance is the relation between the objectives of the ICA and the design of the instrument. Here again the ICAs present a somewhat different picture than many other analytic products. The objectives of an ICA are very clear and the results framework for achieving those objectives is benchmarked by the follow up ICAs in the series. In this context the first ICA in the country becomes a baseline, and the second ICA is used to track improvements in the investment climate over time. An important part of the second ICA is therefore to provide inter-temporal comparisons. While these are present in all ICAs the presentation often lacks the clarity needed for an assessment of IC outcomes. 5.6 Bangladesh: Strategic relevance is rated Satisfactory for the first ICA, but Moderately Unsatisfactory for the second. Had this review been done in 2008 during the interim government the rating for relevance and ownership might even have been rated Highly Satisfactory. The Interim Government had internalized the discussion of the ICA and put in place an excellent institutional framework in the Better Business Forum that could serve as a counterpart for the ICA and provide real ownership and feedback on its recommendations. Two years later, the reality is a Government that has distanced itself as much as possible from the Interim Government and has allowed the institutions it created to disappear and has shown little interest in the ICA recommendations or their follow up. The issues are first, could the Bank have done more to build ownership of the second ICA and the process as a whole, and second whether there is ownership within the Bank of the recommendations and the follow up in the Bank‘s own analytic work and particularly the Bank‘s development policy lending. On the first, while the Bank could not have foreseen that the Prime Minister would be so antipathetic to the IC agenda because of its links with the Interim Government, it could have delayed the second ICA until a new Government was in place and tried to ensure that it would be seen as a new exercise and a joint product. On the second issue, in the view of most interlocutors the relevance and impact of the ICAs in Bangladesh will depend on the willingness of the Bank to put its money where its mouth is on investment climate issues. Some of those interviewed felt that this had not happened and that the Bank in some sense was underplaying its hand in Bangladesh on these issues. 5.7 Egypt: Strategic relevance of the 2004 ICA is rated Highly Satisfactory mainly because of the timing and the presence of the new Minister as a champion for the use of the analysis. The Minister saw the first ICA as providing a baseline on the regulatory environment, and the subsequent ones as providing a monitoring tool. It was largely at his urging that the ICA was undertaken at two or three year intervals. The Minister himself went through the questionnaire before it was approved, and the ministry sent a letter to the firms selected asking/instructing them to cooperate with the survey team from the American 19 The ICA update undertaken in Egypt in 2006/7, which produced a short report at low cost, is an interesting model of such an approach. Even though it reflected a great deal of interest, rather than a lack of interest on the part of the authorities, it was recognized that the proximity to the preceding ICA meant that it was unlikely to yield major new findings, and that its value lay in validating that the reforms were on track. 25 University of Cairo. The Bank provided training along with the ICA both to the institute that prepared the survey, but also to the Ministry which lacked staff with a suitable technical background. Training was provided on how to read the findings. Strategic relevance for the second ICA is rated Satisfactory. There was still genuine ownership and use of the ICA for benchmarking, but by this time, it was becoming a more routine process and there were real questions as to the utility of the instrument going forward. This has probably changed now given the impact of the evolving Egyptian political transition on the investment climate. 5.8 Guatemala: The first ICA is rated Highly Satisfactory for its strategic relevance. From the Guatemalan viewpoint it achieved its main goal of raising awareness. It identified clear priorities and critical themes. Before the ICA was done, Guatemalans used the World Economic Forum (WEF) indicators as their reference point. After its completion, the ICA generated much interest. The ICA produced more detailed information than the WEF and Guatemalans saw its indicators and themes as more relevant than those of the WEF. Moreover, the ICA managed to get the collaboration of relevant actors in Guatemala. The ICA was an integral part of the Bank assistance strategy, had a well-defined rationale and the client was engaged. 5.9 The second ICA is rated Moderately Satisfactory. Following the success of the first (2004) ICA, the Government requested a new ICA as a way of making progress assessment on reforms. In the event, however, the delivery of the new report was delayed by more than a year. By the time the report was ready, a new Government was in power. The investment climate was no longer a major priority. Despite the similarity of contents and quality, the 2008 report did not generate the same degree of ownership as did the previous report. 5.10 Kenya: The 2004 ICA is rated Moderately Satisfactory for strategic relevance. The ICA was a timely product for measuring and analyzing constraints to business, informing the 2004 CAS, and providing input into the Bank‘s 2005 growth report. It addressed a well- identified development constraint, namely the long-standing low international ranking of Kenya‘s investment climate. Its findings and recommendations were potentially useful for both the government and non-government organizations, including other donors. Government engagement was muted on the ICA, with feedback provided only by the direct counterpart, the Ministry of Industry but possibly stronger on the 2005 report to which the ICA provided inputs. The design was appropriate, as it a) coupled survey results with other indicators to identify key constraints with Bank sector knowledge to focus recommendations and b) was to be used in a broader 2005 Bank growth report. The survey was limited to formal manufacturing firms. 5.11 The 2008 ICA is rated Satisfactory. It was timely, providing an updating of the monitoring of constraints to business, following the 2004 ICA and for providing input into the 2008 CAS and the 2008 growth report. As with the earlier ICA it addressed a well- identified development constraint and providing findings of potential use for the government, private sector, NGOs and donors. Government and non-government engagement was strong in providing feedback on the report. The design was somewhat broader and more costly than that of the 2004 ICA. It had a wider coverage, including services, manufacturing and micro- enterprises in addition to formal establishments. 26 5.12 Vietnam: Strategic relevance is rated is rated Moderately Satisfactory. The Bank‘s initiative for the ICA and the preparation of the report took place at a time when the Vietnamese economy was rapidly opening up from a state-directed model towards a market economy, and when there was healthy momentum on the side of the government to address obstacles in the investment climate. The ICA‘s main focus of collecting and presenting perceptions of the level playing field between state-owned and private firms was a highly relevant initiative at the time. This topic, and others covered in the ICA were identified in the Bank‘s Country Assistance Strategy, lending and non-lending, and in the dialogue with the government and donors. This makes for a high relevance ex-ante, but the relevance ex-post was negatively affected by the dimension of timing. The ICA was quite delayed. The ICA‘s concept note was drafted in the second half of 2004, the final report is dated November 2006, and the government‘s comments were received late 2007, at which point in time, the Ministry of Planning and Investments (MPI) plausibly made the point that during the preceding two years many regulations were improved to create a more favorable climate for doing business. Part of this tardiness can be explained by the change of TTL after the first draft of the report was prepared. The passage of the Investment and Enterprise laws was expedited by the campaign for WTO accession. The ICA lost much of its relevance. 5.13 The case of Vietnam illustrates the need for intensive client engagement to build ownership of the ICA process. There are two dimensions that emerge from the case study. First, engagement and collaboration were concentrated at the beginning and tail end of the process. As this process was lengthy, it meant that the MPI, the ICA‘s main counterpart, was left out of the drafting process and could comment only at a late stage. Second, client engagement could have involved broader collaboration. The ICA covers a variety of topics, many of which fall outside the direct responsibility of the MPI. On top of that, capacity at MPI is limited. Greater involvement from other ministries or other (semi)private organizations could have increased ownership and could have provided a more direct venue for follow-up. With the benefit of hindsight, the absence of such an approach of broader engagement contributed to the report‘s ‗sudden death‘ as soon as it was delivered to MPI. 5.14 The ratings on relevance are shown in the Table 6 below: Table 6. Ratings on Relevance ICA Rating Bangladesh 2003 Satisfactory Bangladesh 2008 Moderately Unsatisfactory Egypt 2004 Highly Satisfactory Egypt 2009 Satisfactory Guatemala 2004 Highly Satisfactory Guatemala 2008 Moderately Satisfactory Kenya 2004 Moderately Satisfactory Kenya 2008 Satisfactory Vietnam 2006 Moderately Satisfactory Note: These ratings are the evaluators‘ judgments regarding the outcomes of each ICA. They are not an assessment of the performance of the Bank, the task team or the Government. 27 5.15 The differences in relevance ratings across countries are largely explained by the variance in ownership. In both the cases where the ICAs were rated Highly Satisfactory for relevance (the first ICAs in Egypt and Guatemala, there was a Government or a Minister willing to ‗champion‘ the ICA – a keen audience for the diagnostics with an interest in the ICA as a means for benchmarking progress. While it could be argued that this is largely a matter of good luck rather than good judgment, some of the country studies suggest that the Bank could have done more in other cases to create broader interest in the ICA either by delaying the preparation until a new Government took office, or by involving a broader sub- set of stakeholders in the preparatory discussions. The Vietnam case also makes clear the importance of timing and particularly of the value of an interim briefing on the basis of the preliminary survey results rather than waiting until the final report. 29 6. Quality of the ICAs 6.1 As an instrument of quality control, consistency of data across countries, and knowledge sharing, DEC and FPD have created a panel of experts on the ICA instrument. Some of the reports covered by this PAR were prepared by members of the panel and there is a requirement that at least one panel member should be included in the list of peer reviewers for each ICA. In addition panel members provide advice and input into the ICA preparation, particularly on issues relating to the survey design. 6.2 This quality assurance has been important in ensuring that the surveys provide a useful and reliable diagnostic of the key investment climate constraints. This said there are some issues on the reliability of the cross-country comparisons and the inter-temporal comparisons. The problems with the survey often relates to the poor quality of the data base provided by the statistical service of the country from which the sample is drawn. These often do a very poor job of capturing the enterprise population. They also of course do not include informal enterprises. As a consequence of these problems, the ICA was initially designed to cover manufacturing sector enterprises with more than 10 workers. In many countries, Governments and Bank staff have been dissatisfied with such limited coverage and the ICA‘s have been extended to cover the services sector as well as manufacturing and also to cover smaller and informal enterprises. The ICA team usually comes out for the launch of the survey. The data is then tabulated and analyzed and the report prepared. 6.2 For purposes of this performance assessment report (PAR), the analysis of technical quality relates to the quality of the ICA report and not to the quality of the enterprise survey, which provides the empirical basis for the ICA.20 For purposes of analyzing the quality of the ICA, the case studies reviewed: (i) the presentation of findings – to what extent do they provide clear and coherent messages on the diagnostic – and; (ii) the quality of the recommendations – to what extent do they follow from the diagnostic and are they generic or actionable. Both the presentation and recommendations reflect the collective knowledge and skills of the country team, including sector specialists whose work and expertise may be drawn upon by core members of the ICA task team. 6.3 The value-added of the FPD Network is often created at this juncture through the reviews by the panel of experts which play a key role in the quality of policy advice. While firms‘ perceptions obtained through surveys are crucial for identifying constraints, the actions needed to address the issues must be based on broader considerations. For example, taxation is often a concern to private firms, but easing the tax burden needs to be done with great care to limit the risk of fiscal imbalances. The panel of experts provides a checkpoint for ensuring that the trade-offs of major costs and benefits are adequately evaluated. In addition, the country team also plays a role in quality assurance by adding depth and the experience of overall country dialogue in formulating or screening policy proposals. 20 As noted in Chapter 2, the enterprise survey is an independent donor funded program managed by the Bank within the FPD Network. It has its own staff and work program independent of ICAs. For a given country, the enterprise survey is conducted more frequently than is ICA. 30 6.4 While technical quality is a strong dimension of the sample ICAs, there is scope for improvements. One cross-cutting issue is the limited depth on institutional analysis – and the political economy – of reforms. A better grasp of the hierarchy of decision-making authorities would further enhance the credibility of policy advice, as well as the likelihood of achieving the desired objectives. Furthermore, a more systematic approach to the use of available sector knowledge and ongoing policy dialogue would also strengthen the quality of recommendations in most of the sample ICAs. 6.5 Bangladesh: The technical quality of the first ICA is rated Satisfactory. It was one of the pilot assessments. It set realistic goals and achieved them adequately with a very clear and prioritized presentation. The second ICA is rated Moderately Satisfactory. It was far more ambitious incorporating a huge module on the rural sector. The scope of the questionnaire required multiple visits and took 3 to 4 hours of senior executives‘ time to complete. The analysis is sophisticated and of high quality, but the report is so laden with comparative data – cross-country, inter-temporal, spatial, sub-sectoral – that it becomes almost impossible to wade through one dense paragraph after another. The initial report was largely diagnostic in nature, but the CD insisted that it include operational recommendations. The report drew on the considerable body of existing sector analysis for this purpose. Although this is not a common practice for the sample ICAs, the recommendations were sound and helped advance the Bank‘s policy dialogue. Box 3. The Climate for Investment or for Operations? The enterprise survey can cover only existing firms and not investors interested in entering business. But the issues faced by prospective investors may be different from those of existing firms. In Bangladesh, for example, the survey had a counter-intuitive finding that few businessmen found access to land a serious constraint. The ICA authors rightly discounted this as representing the views of those businessmen who had already set up their firms and had their plant and equipment in place, and who were not looking to expand their factories. Similarly, in a number of countries the surveys tend to understate the issue of access to power. Most mature firms have their own power generators and do not depend on a regular supply of electricity from the national grid. Yet for new businesses, the added capital and operating costs of purchasing and running power generators may make the difference between going ahead or not. 6.6 Egypt: The technical quality of both ICAs is rated Satisfactory. The ICA was viewed as an important diagnostic tool that gave rise to a variety of more narrowly targeted analysis. Among the follow-up studies are: a much more detailed report on land policy; a FIAS review of economic zones; a huge USAID project on regulatory reform; and work by IFC on corporatization of state enterprises. Many of these follow up documents use information drawn from the ICA. 6.7 A particularly interesting feature of the 2009 ICA was the inclusion of questions on the availability of skilled labor. This was not generated initially from the firm interviews, but was a consequence of the Bank‘s sector work on education and vocational training. As a consequence the ICA looked at this issue and validated the concerns that Egypt‘s education system was not geared to the quantity and quality of vocational skills needed by the 31 expanding business and services sectors. This is a very good example of effective synthesizing of analysis from different sectors – often a weakness in the Bank. 6.8 The ICAs are very well presented, readable with clear graphics and well-chosen comparisons that do not overwhelm the reader. It was evident from discussions in Cairo that there was limited interest in international comparisons. The real concern here was to track the improvements in perceptions over time. Given the good technical quality, it is even more disappointing that the reports have not reached a wider audience. 6.9 Guatemala: The 2004 ICA is rated Satisfactory. It was held in high regards for its credibility and technical quality. These characteristics made the findings a suitable benchmark for monitoring the competitiveness agenda. The report served to show where Guatemala was in terms of investment climate and how it compared to similar countries. In- country stakeholders noted, however, that (a) the general nature of the recommendations did not help them on specific actions and (b) policymakers did not have the time to read the entire report. The general recommendations point to a lack of knowledge of the institutions, as well as a weak grasp of the political economy. The latter was not a serious constraint for the 2004 ICA because those monitoring the competitiveness agenda were able to supply the missing ingredients. 6.10 The technical quality of the 2008 ICA is also rated Satisfactory. Although the 2008 ICA was strong on its credibility and quality, the report did not shape the competitiveness agenda because the incumbent government gave low priority to the subject and, in any case, preferred to work with IDB, which had presented more comprehensive reports on the issues. Guatemalans also noted that, despite its quality, the report was not well suited to the country because it was long and not available in Spanish. Its recommended policies were also too general. These problems contributed to the report‘s low profile and modest influence. Many interlocutors recommended offering different versions of the report for different audiences. 6.11 Kenya: The technical quality of the 2004 ICA is rated Satisfactory. The thrust of the 2004 ICA is that (a) the competitive disadvantage of Kenya‘s manufacturing firms vis-à-vis strategic competitors like China and India is due to outdated plant and equipment and comparatively costly labor; and (b) productivity growth has been zero or negative. Outdated equipment reflected decades of low investment levels. Excessive labor costs may have resulted from escalations in mandated minimum wage and public sector pay. In addition to high labor costs, high levels of crime and corruption and poor infrastructure contributed to an adverse investment climate. 6.12 The technical quality of the 2008 ICA is rated Satisfactory. It pursued the same broad themes – low productivity and competitive disadvantage. The report indicated that the productivity of Kenyan firms improved during the 2003-07 period – primarily as a result of higher capacity utilization, but it remained below that of firms in other middle income Sub- Saharan Africa. As with the preceding period, corruption and deficient infrastructure (especially roads and power) remained the key constraints. Both reports made some references to political economy issues and identified political instability as a constraint. Recommendations were mixed, ranging from clear and actionable ones to broad 32 generalizations. The second ICA showed an improvement over the first in terms of presentation and clarity of the key findings. 6.13 Vietnam: The technical quality is rated Moderately Satisfactory. The Vietnam ICA is an example of a report that serves as a diagnostic tool, rather than providing a roadmap for reform which would require sequenced and prioritized policy actions. Recommendations are implicit and embedded in the discussions. One of the key themes is the need to provide fair competition between private businesses and state owned firms. In addition, there is a chapter dealing with regulations and corruption, which many officials regarded as not sensitive. Some of the key issues, including the conditions of major infrastructure such as the ports, did not get the attention they deserved. Overall assessment 6.14 The ratings for technical quality are shown in the table below: Table 7. Ratings for Technical Quality ICA Rating Bangladesh 2003 Satisfactory Bangladesh 2008 Moderately Satisfactory Egypt 2004 Satisfactory Egypt 2009 Satisfactory Guatemala 2004 Satisfactory Guatemala 2008 Satisfactory Kenya 2004 Satisfactory Kenya 2008 Satisfactory Vietnam 2006 Moderately Satisfactory Note: These ratings are the evaluators‘ judgments regarding the outcomes of each ICA. They are not an assessment of the performance of the Bank, the task team or the Government. 6.15 In general, the ICAs are well up to the Bank‘s high standards of technical quality. In all cases the survey results provided good diagnostics and the reports provided clear analysis of the core constraints to private investment. The two problem areas that have been identified are the quality of; first, the presentation and second, the recommendations. 6.16 On presentation, the ICA surveys provide enormous amounts of data and it is all too tempting for TTLs to present as much of it as possible. In some reports, paragraphs are crowded with comparisons, often pointing in different directions. For example, cross- country comparisons may be favorable but the inter-temporal comparisons may show a lack of progress on the issue. Careful discussion that enables the reader to assess the severity of the issue is seldom seen. As the ICA cycle progresses in most countries, the reports seem to expand in coverage and become longer. The first Bangladesh ICA is concise and has a very clear set of messages. The second is a lengthy and complex document, closer to a source of reference than a focused piece of policy analysis. 33 6.17 The quality of the recommendations is cited as an issue in most of the case studies. The enterprise survey yields diagnostic information, but by design it does not include any analysis of the likely impact of reform measures. This has to be drawn from the sector analysis that the Bank carries out in the relevant areas. The involvement of the country team and the review panel is therefore a key element that can help improve the quality. 6.18 Specific recommendations generally require that the Bank carry out sector work before or after the ICA. Even where the ICA simply takes recommendations from current Bank sector work it is difficult to provide sufficient context and logic to make them convincing. The approach taken in Egypt was simply to indicate the kind of actions that were needed and let the Government, with selective support from Bank sector work, formulate its own follow up programs. This allowed the Government to factor in political economy considerations and determine the best sequencing of measures. The ICA played its role of benchmarking progress on each of the constraints identified. 35 7. Dialogue and Dissemination 7.1 IEG research shows that the quality of dialogue on knowledge products is highly correlated with the effectiveness.21 On this front, ICAs face a demanding test, since by design it involves a large number of stakeholders both in the public and private sectors. Even excluding to the firms that participate in a large scale survey, the interested parties still include business associations, advocacy groups, civil societies, in addition to the line ministries and agencies providing economic services. 7.2 To achieve the central objectives of informing the public and motivating reforms, the ICA task teams have generally reached out to a broad audience. Initially, the goal is to raise awareness of the work envisaged and encourage participation. Stakeholders are often given an opportunity to influence the approach. In the case of Egypt, for example, the Minister requested that the ICA coverage of Upper Egypt be expanded and the Bank agreed. In the case of Bangladesh, there was interest in extending the survey to enterprises in smaller towns and rural areas. Early consultation is particularly important among line agencies which might play a role in implementing the findings, as well as local research institutes which might be able to contribute to the work. In the case of Guatemala, the Competitiveness Commission – an agency in charge of reforms – worked closely with the task team. In Kenya and Vietnam, a local firm was chosen to conduct the enterprise survey. 7.3 Once everything is in place for the survey to be carried out, there is generally a workshop with press coverage. This is important because the survey requires a considerable expenditure of time by owners or senior executives (in the case of the Bangladesh ICA it was estimated that 3 to 4 hours were required from each firm surveyed). An official launch therefore becomes a requirement to ensure that enterprises that are selected cooperate with the survey. In the case of Vietnam and also Bangladesh, the Chamber of Commerce stressed to its members the importance attached to their cooperation, whereas in the Egyptian case the Government sent a letter to the survey participants asking for their cooperation. 7.4 The dialogue reaches a peak during the dissemination stage. Once the ICA report is prepared, there is generally a high-profile dissemination event to explain the key findings. Government, donors and private sector representatives are generally invited, along with the media, civil society and the general public. In most cases the full report is disseminated at these events, but in Egypt and Vietnam only PowerPoint presentations were provided. 22 After the launch events, however, engagement efforts often dissipate rapidly. Follow-up activities remain uncommon. Bank staff in field offices, who are in contact with clients on a daily basis, see the need for more systematic follow-up to resolve unanticipated issues or to provide further assistance, including pursuing new loans to address the issues. 7.5 The ICA survey generates a very interesting data set with a great deal of potential value to in-country researchers. But in practice the case studies found almost no instance where the actual data were being used for purposes other than the ICA preparation, especially 21 IEG (2008). Using knowledge to improve development effectiveness: An Evaluation of World Bank Economic and Sector Work and Technical Assistance, Chapter 5. 22 It is probably not a representative sample that in two of the five cases examined by IEG, the Government declined the Bank‘s request to publish the ICA – generally by not saying ‗yes‘ rather than actively opposing it. 36 given the weaknesses of the official industrial censuses in most countries. Where special surveys of the informal sector or of rural non-farm enterprises were carried out, these are often the only source of data on these important economic categories. All the country data is available on the Bank‘s external web-site, yet this fact seemed unknown to most researchers in the academic institutions visited by the missions. Dialogue in the countries covered 7.6 To evaluate the quality of dialogue and dissemination, this PAR focuses on the impact of the ICA on the awareness and opinions of the target audience. Evidence of adequate dissemination is particularly important, including whether or not the report is made publicly available and by what means. Some of the key public officials and stakeholders, however, require more than access to the publication. To reach the right audience, the task team may need to undertake closer engagement including direct briefings and discussions. In addition, this PAR considers the extent to which the dialogue is sustained to allow for meaningful communications between the Bank and its local partners. 7.7 Bangladesh: The dialogue and dissemination is rated satisfactory for both ICAs. The task teams undertook effective consultations and launch events with a broad range of stakeholders. In addition, new organizations dedicated to improving investment climate were created for follow-up and sustained dialogues. 7.8 The first ICA was launched at a major event and included cabinet-level representatives, as well as guest speakers from India, Pakistan, and Sri Lanka. Working groups were organized on power, telecommunications, ports, business regulation, and accountability of the civil service. The message that GDP growth was being undermined by interruptions in power supply confirmed what was already well known. The formation of the Bangladesh Investment Climate Fund (BICF) provided an effective follow up instrument. 7.9 The communication strategy of the second ICA was elaborate and well thought-out. Aiming for best practice, the ICA team consulted with the Bank‘s External Affairs Department and used the Turkey ICA as a benchmark. Launch events were organized in Dhaka and Chittagong as collaborative efforts with the Board of Investment and Chamber of Commerce. These were well received with considerable press coverage. However, the Bank organizes many dissemination events in Bangladesh and few of the participants in these workshops can actually remember participating in this particular event. In addition, the ensuing policy dialogue proved to be fragile as the new government disbanded the Regulatory Reform Commission (RRC) and the Better Business Forum (BBF), which formed the main vehicles through which the private sector engaged the official sector. 7.10 Egypt: The dialogue is rated moderately unsatisfactory for both the 2004 ICA and 2009 ICA. The basis for these ratings is the failure to publish the ICAs. The first ICA was only disseminated in Cairo through a one-day event held for selected representatives of the Government. The dissemination was by a slide presentation and done before the report was finalized. According to the Government, the first ICA was not made public because it contained ―some value judgments that are not substantiated‖. The ICA, however, was widely shared within the Government including with the Prime Minister and the Economic Committee. But the final report was not published. This has to be understood in the 37 Egyptian context where the Government is often reluctant to release information. In some cases, information is given to selected Bank staff on condition that no other staff shall have access. Under these circumstances, the report could not have contributed to public debate. 7.11 For the 2009 ICA, a large dissemination event was held, with the Minister of Investment attending. In addition, there was dissemination outside Cairo. Once again, this was all on the basis of PowerPoint presentations. The final report was not made available to the public. However, the fact that the TTL was based in Cairo made it possible to carry out broader dissemination activities. In addition, it was possible to sustain the dialogue and provide follow-up assistance. 7.12 Guatemala ICAs. The dialogue on 2004 ICA is rated satisfactory; but that of the 2008 ICA is moderately unsatisfactory. The communication strategy is similar in the two cases, but the first ICA benefited from an unexpected surge in interest due to the onset of negotiations on the Central America Free Trade Agreement. The task team was able to engage major stakeholders expeditiously. The second (2008) ICA had no such luck. 7.13 The 2004 ICA benefited from a successful launch event which attracted good media coverage. The Bank presented the findings of the report to key public officials and agencies such as Competitiveness Commission which was in charge of the reform agenda, and the Consejo Ampliado de Competitividad, a public-private advocacy group. The report was published and made available on the web. Public access to the ICA, however, was constrained somewhat by the absence of a Spanish version or a self-standing summary, which would have been welcome by the local audience. 7.14 The 2008 ICA was presented only to the Bank‘s main counterparts in the country. Although the report was published, no launch events or broad dissemination to the general public were undertaken. And like the 2004 ICA, the report was not available in Spanish or in abridged versions. Government officials cited the contrasting example of the Inter-American Development Bank which made its reports available in Spanish. Some interlocutors pointed out that the Bank‘s limited engagement on the investment climate weakened the dialogue. 7.15 Kenya ICAs. The 2004 ICA is rated moderately satisfactory and the 2008 ICA satisfactory. The dialogue and dissemination process of the first (2004) report involved the counterparts in the Government and private sector representatives. The interlocutors, including the Kenya Association of Manufacturers and researchers in the Ministry of Trade, knew the report well and made use of the findings. However, most of the stakeholders mentioned in the ICA could not recall the report. This may reflect the passage of time (6 years) and the fact that the task team included mainly consultants and not regular staff with broader engagement in the country. 7.16 For the second ICA (2008), major efforts were made on dissemination and outreach. The media coverage was extensive. The report itself provides a good summary and useful graphics to convey the findings. It appears to have reached the right audiences including many vocal private sector advocacy groups. However, the dialogue has been relatively brief, with few follow-up activities taking place since the launch events. 38 7.17 Vietnam ICA. The dialogue is rated unsatisfactory. Although in the early stages close consultations took place with key agencies in the Government, including the Ministry of Planning and Investment (MPI) and the Prime Minister‘s Advisory Group. But the dialogue came to an end upon the departure of the TTL in late 2005. The final report was not published or disseminated. Before her departure, the TTL discussed a draft of the report with MPI officials, and the second TTL delivered the final draft in November 2007. There is no evidence that MPI shared the ICA report with any agency outside of the ministry. 23 Overall assessment 7.18 With some exceptions, the dialogue on ICAs has been conducted well to advance the goal of knowledge creation and sharing. In some cases, the country dialogue has been constrained by local political directives, but these restrictions do not necessarily preclude the effectiveness of ICAs. Where possible, the Bank has attempted a broad outreach to stimulate public debate, in addition to collaboration with the main counterpart. 7.19 There is, however, room for improvement.  For one thing, it is desirable to expand early consultations and engagement with local partners, including researchers and donors. This would help make the final product more relevant and facilitate the objective of building local capacity.  In addition, effectiveness could be enhanced if the high-profile launch events were to be accompanied by additional follow-ups and consultations. These activities would allow the Bank to monitor results and identify opportunities for further support.  Finally, there is scope for making the Bank‘s communication strategy more adaptive, capable of responding to local conditions. For example, the dissemination efforts could be scaled up or down, perhaps substantially, depending on the receptivity of the audience. Furthermore, the shape and form of reports might be adjusted to suit local expectations, including using shorter versions in the local language. 7.20 Of the nine reports evaluated, five are satisfactory or moderately so. Four are rated less than satisfactory. See Table 8. Table 8. Ratings for Dialogue and Dissemination Report Rating Bangladesh 2003 Satisfactory Bangladesh 2008 Satisfactory Egypt 2004 Moderately Unsatisfactory Egypt 2009 Moderately Unsatisfactory Guatemala 2004 Satisfactory Guatemala 2008 Moderately Unsatisfactory Kenya 2004 Moderately Satisfactory Kenya 2008 Satisfactory Vietnam 2006 Unsatisfactory Notes: These ratings are the evaluators‘ judgments regarding the outcomes of each ICA. They are not an assessment of the performance of the Bank, the task team or the Government. 23 Nonetheless, the main findings of the report were presented in the 2006 Vietnam Development Report, which was widely circulated. 39 8. Findings, Issues and Suggested Approaches 8.1 This PAR was not designed to assess the value of the enterprise survey which the Bank uses to obtain comparative country data. The focus is on the value of the ICA report as an instrument of Country Economic and Sector Work in advancing the Bank‘s country strategic agenda. The overall assessment of this PAR is that the ICA has played a useful role in selected situations and that the role could be enhanced if the Bank were to analyze more carefully the political economy and the likely receptiveness to ICA findings and recommendations and calibrate the instrument more carefully to those situations. a) IS THE ICA DIAGNOSTIC OR PRESCRIPTIVE? All the case studies refer to the unsatisfactory nature of the recommendations - excessively generic, off-the-shelf, too general to be operationally useful etc. This raises a fundamental question about the nature of the ICA. The survey provides a diagnostic - it points out the priority areas to be addressed for improving the investment climate. The team is usually constituted by FPD and consists of staff whose skills relate to that sector or to survey preparation and analysis. However, they are not in a position to provide detailed recommendations on how to address the energy deficit, or improve the functioning of the land and labor markets, etc. Either they provide very general advice that goes little beyond indicating that something needs to be done, or they rely on Bank colleagues to come in and provide separate advice. One solution might be to prepare and present the ICA primarily as a diagnostic exercise. It can certainly include a policy matrix, but needs to make it quite clear that these are general recommendations intended to give a sense of priority actions, and are not intended to encompass specific operational steps. Part of the ICA discussion needs to be a decision by the Bank and the Government on the follow up analysis required to make these general recommendations operational. The ICA can certainly be used to highlight those areas where the Bank or others have already done significant policy work and an agenda for action already exists. In other areas, the Bank may need to support TA or mount follow up missions to help the government design an operational program for addressing the issues identified in the ICA. b) SHOULD THE RESULTS OF THE ICA BE JUDGED ON THE BASIS OF INDIVIDUAL ICAs OR AS A 'CYCLE' OF MULTIPLE ICAs? The dilemma is that the results of the ICA – in all cases – depend on the attitudes of the incumbent Government at the time the report is delivered, regardless of the degree of ownership at the launch of the ICA. For these reasons, the first ICA in Guatemala and Bangladesh had good results and the second ICA less so, while in Kenya this pattern was reversed. The ICA might need to be conceived and planned as a cycle, with the survey undertaken at regular intervals with the key diagnostic being the changes in perceptions of the business climate during the intervening period. It would thus complement the DBIs in demonstrating how the private sector sees the issues. The DBIs are often criticized for reflecting the opinions of experts, basing their judgments 40 on legal requirements, rather than reflecting the practical conditions that face entrepreneurs. The ICAs provide a valuable correction. The frequency of the ICA depends on the country situation. In Egypt where intensive reforms were being undertaken, the Minister wanted a two-year frequency. In other countries, three to five years seems more appropriate, but it should be specified up front and the frequency maintained whatever the attitude of the Government. The real 'result' of the ICA is whether the investment climate is improving and this is measured by the next ICA in the cycle. Clearly, an objective of the first ICAs in a country is to provide a baseline against which performance can be measured. c) IS IT ESSENTIAL TO ENSURE THE PRESENCE OF A CLEAR 'CHAMPION' FOR ICA IN THE GOVERNMENT? The Egyptian experience suggests that the interest of a key minister or senior government official can be a sufficient condition for the success of an ICA. The Bank could more consciously target a particular Ministry and develop the ICA as a joint product which can be used both inside Government as a means of coordinating other ministries around the IC agenda. Furthermore, there are indications that, in the absence of a clear champion, it might nonetheless be possible to cultivate one through intensive engagement and close partnerships with targeted agencies. d) WHY IS THE RECEPTIVITY OF ICAs MORE VARIABLE ACROSS COUNTRIES AND OVERTIME THAN OTHER BANK DIAGNOSTIC WORK? It may be that some Governments do not consider growth and investment – the focal points of ICAs – a high priority. Or it could be that the ICAs deal with sensitive and politically volatile issues of political economy and governance and amplify the voice of the private sector, which some governments are not keen to listen to. Other core knowledge products of the WBG, such as the public expenditure review or country economic memorandum, are more firmly established within the overall IFI reporting system and are therefore more difficult for governments to reject or ignore. However, it is more important to recognize the fragility of political receptiveness than to fully understand the reason why. This recognition helps the task team anticipate the risk of losing relevance when a transition in the Government is imminent. e) WHAT IS THE BEST WAY TO DISSEMINATE AN ICA? Planning ICA dissemination is a very difficult task. In Bangladesh, a very detailed and careful dissemination was undertaken with large high level meetings and considerable press coverage. This does not seem to have resulted in much retention of the key messages of the ICA. In Vietnam and Egypt the ICA, the dissemination was restricted. Only in one or two cases were the summary and conclusions of the ICA translated into a local language and even in these cases the translated version was not made publicly available. A few principles might help:  First, at the very least a four page summary of the ICA should be prepared for release both in English and the local language. 41  Second; there should be up-front agreement with the Government that while the Bank will be happy to sanitize the ICA to reflect Government comments, that it will be posted on the Bank web-site and therefore publicly available.  Third, private firms have voluntarily given the Bank their time to complete the survey. The Bank owes them a feedback. The dissemination process should include these firms. f) SHOULD OTHER DONORS BE INVOLVED IN THE ICA PROCESS? One of the general impressions of this review is the absence or marginal involvement of other donors with the ICA process. In many cases, ICAs could be an important tool for donors interested in private investment and competitiveness issues, particularly the regional development banks, DFID, USAID, OECD, etc. In a number of cases, other donors mounted quite major activities that overlapped with the ICA and even carried out their own surveys, including the OECD in Egypt. One of the recommendations would be a more systematic involvement of other donors. g) HOW SHOULD THE VALUE OF THE SURVEY BE MAXIMIZED FROM A CAPACITY BUILDING PERSPECTIVE? It would be desirable to ensure that extra mileage is obtained from the periodic surveys conducted by the Bank. The following questions might be incorporated in the planning of survey:  Is the survey conducted in ways that built capacity in the country to carry out similar surveys in future?  How does it relate to the statistical services of the country?  Is the data reliable?  Is the data set resulting from the survey made available to local researchers? The results of the enterprise survey are available through the Bank web-site, but in some of the countries this valuable resource is not widely known.24 The dissemination of ICA – where it is not prohibited – provides an excellent opportunity to reach the right audience for the purpose of promoting the enterprise survey and the website. Over time, this would also encourage the researchers to participate more actively in the preparation of ICAs.25 24 The URL of the website is: http://www.enterprisesurveys.org/ 25 According to FPD staff, to improve the quality of survey data, greater efforts are being made now to assist in the capacity building of local contractors and national statistical offices in some cases. 43 ANNEXES Disclaimer: The following Annexes contain documents that served as preliminary inputs for the ratings presented in this report. They reflect initial findings, as well as notes and worksheets of individual evaluators. During the review process, these initial results are modified and revised to ensure consistent applications of the methodology. The final assessments as shown in the main report may thus differ – at times significantly – from those given here. These Annexes are included for the sake of transparency. ANNEX 1: BANGLADESH ............................................................................................ 45 ANNEX 2: EGYPT........................................................................................................... 61 ANNEX 3: GUATEMALA .............................................................................................. 75 ANNEX 4: KENYA ......................................................................................................... 93 ANNEX 5: VIETNAM ................................................................................................... 117 ANNEX 6: LIST OF PEOPLE MET .............................................................................. 125 45 ANNEX 1: BANGLADESH Performance Assessment Report on Investment Climate Assessments Bangladesh Investment Climate Assessment 2003 Vice President Nicholas H. Stern Country Director Christine Wallich Sector Director Sadiq Ahmed Sector Manager Joseph Del Mar Pernia Task Leader David R. Dollar Project ID/ Type/ Report No./Cost P077465/Strategic Task/ 33689/$161,000 Number of: Volumes/ Pages 1 volume / 81pages Languages English Document Type Investment Climate Assessment Topics Private Sector Development Unit Finance & PSD Sector Unit (SASFP) Bangladesh Investment Climate Assessment 2008 Vice President Isabel Guerrero Country Director Xian Zhu Sector Director Ernesto May Sector Manager Simon Bell Task Leader Tatiana Nenova Project ID/ Type/ Report No./Cost P099838/Strategic Task /56635/$558,000 Number of: Volumes/ Pages 1 Volume /89 pages Languages English Document Type Investment Climate Assessment Topics Private Sector Development Unit Finance & PSD Sector Unit (SASFP) Background Two Investment Climate Assessments (ICAs) have been produced for Bangladesh. The first, prepared in 2001/2 and published in 2003, was one of the first of its kind and had a significant input from the central unit involved in developing the survey and the basic approach. A major launch event was staged with participation from senior Washington managers and high profile press coverage. The message that Bangladesh was losing two percentage points of GDP growth each year because of inadequacies and interruptions in power supply was especially resonant. The focus of the first ICA was essentially diagnostic with the policy recommendations implicit in the identification of four key issues (infrastructure, particularly power; access to finance, particularly for small and medium enterprises; bureaucracy and red tape; and corruption). The scope of the enterprise survey was limited to the cities of Dhaka and Chittagong, and only covered manufacturing enterprises with more than ten workers. The ICA, together with the ‗Doing Business‘ surveys, moved the discussion of investment climate issues to center stage in Bangladesh and this contributed to a decision by the UK‘s Department for International Development (DFID) and IFC to establish a Bangladesh Investment Climate Facility (BICF) in 2006 to support Government efforts to improve the investment climate particularly with respect to improving the regulatory environment, reducing bureaucratic interventions and delays, with the expectation that this would also yield improvements in governance. At the beginning of 2007 with the domestic political situation becoming increasingly chaotic, the Bangladesh military intervened and installed a caretaker government headed by the then Governor of the Central Bank, Dr. Fakhruddin Ali Ahmed, a former World Bank staff member. The BICF proposed to the new authorities that two new public/private institutions be set up to address investment climate issues: A Regulatory Reform 46 Commission (RRC) and a Better Business Forum (BBF). The caretaker government moved rapidly to appoint these two groups and they proceeded to move into an impressive agenda of new activities to simplify and streamline regulation. In the course of 2006/7 it was decided to undertake a follow up ICA. The scope of the enterprise survey was wider than the first one: this time around it was decided to cover spatial issues as well by extending coverage to peri-urban and rural areas in a larger number of localities. Since most rural firms had less than 10 workers, this also required enlarging the population of small enterprises. In addition, since services and construction were a major share of rural enterprises, these were covered as well. The new ICA was published in 2008. In addition to the confirmation that the issues of the previous ICA remained relevant, two important new issues emerged from the 2008 ICA. The first was the issue of access to land as an increasingly important constraint facing enterprises, and the second was the question of the availability of skilled labor in various sectors. The Better Business Forum formed working groups to study these issues further and, at the request of the Country Director, the ICA team went beyond the diagnostics and developed a set of policy recommendations for Government and donor follow-up. At the end of 2008 elections were held and produced a resounding victory for the Awami League party. The new Government rejected many of the initiatives that had been undertaken by the caretaker government, on political rather than substantive grounds. Among the casualties were the BBF and RRC, despite recognition of the useful role that these two agencies were playing. The reform agenda was left to individual ministries to follow up, and although there was some incremental progress on investment climate issues, most of the bolder strokes that had been proposed were off the table. BICF continues to exist and has shifted its focus to much- needed capacity building efforts, and the Bank continues to take up investment climate issues through its Poverty Reduction Support Credits (PRSCs) which have considerable influence in Bangladesh. Overall Assessment The 2003 ICA is a well prepared report. It piloted the enterprise sampling and survey instrument in Bangladesh and helped build capacity to carry out such surveys. The report is clear and straightforward and it provides useful validation and quantification of ―what everyone knew‖. It was well disseminated and contributed effectively to an expanding interest in Government, private sector and the donor community in the investment climate. The formation of BICF provided an effective follow up instrument for the ICA. The 2008 ICA has had modest impact. While much more ambitious than the 2003 ICA and with some useful new insights and information in areas such as access to land and availability of skills, it does not always make effective use of this new material. It was unlucky in its timing in that a much later survey could well have been prepared in consultation with the new Government and have generated some ownership from it. Its most important contribution appears to have been for the Bank‘s own analytic work and lending program, particularly the PRSCs. The usefulness of the policy recommendations is also open to question since they were not the consequence of serious sector analysis, but tended to reflect off-the-shelf solutions to the problems identified. Lessons 1. The Bangladesh ICA suggests that it is extremely important to think through and discuss at the concept stage what precisely the deliverable is and what kind of results the Bank would like to see flow from it. Since the ICA is based on a standardized survey prepared at intervals, there is a real risk of the ‗treadmill‘ effect with successive ICAs being prepared that provide much the same information and disappear without impact on either the Bank or the country. 2. Most TTLs are eager to design ambitious products that go well beyond the standard data and constitute in effect PSD sector assessments. More consideration should be given however, to the possibility of less ambitious products carried out at relatively low cost, which serve as part of the overall benchmarking exercise, both for the country in terms of measuring progress over the 3 to 5 year interval, and vis a vis comparators. 47 3. As an adjunct of such less ambitious ICAs consideration could be given to carrying out sector work on some of the issues identified by the survey in a follow up phase which could then identify specific policy options for consideration by the Government and the donors. 4. The Bangladesh ICAs also illustrate the lesson that some thought needs to be given to the internalization of the ICA in the country. In many countries it should be feasible to outsource the preparation and write- up of the ICAs to an independent think-tank so that it is seen as a national product rather than a Bank product. The Bank would still presumably need to provide an input to ensure international comparability of the data. 5. A much more explicit part of the ICA process should be consideration of the national capacity to follow up on the issues identified and the need for capacity building efforts by the Bank/IFC in support of both the upstream capacity such as the Statistical Services or the legal frameworks for PPPs, and downstream capacity such as the ability of the bureaucracy to implement regulations. 6. In the specific case of the 2008 ICA where major new components were identified thought should have been given to the possibility of unbundling the rural ICA and issuing it separately. The current presentation downplays the usefulness of the rural ICA and results in paragraphs with so many comparisons in them (international, temporal, spatial, size, etc.) that readability and impact are seriously impaired. 7. The issue of dissemination of ICAs needs to be thought through. The Bank organizes so many launch and dissemination events in a country such as Bangladesh that few of the participants in these workshops can actually remember participating in them. According to interviewees, the dissemination workshop for the 2003 ICA was considered to be one of the best of its kind, with high level presentations, lively discussions, and organization into public/private break-out groups that developed specific actions plans for implementation. Yet even in this case memories were vague (which could be understandable given the time that has passed) and no discernible efforts were made at linking follow up to this dissemination event. 8. It is sometimes difficult to get a clear and simple answer to the question – Is the investment climate in the country improving? – in the ICA documents. The answer is buried in a host of complex tables and explanations. A short introductory statement along these lines is needed. Ratings Summary 2003 ICA Criterion Rating Comments Results Moderately The first ICA had a very promising impact when it was taken up by Satisfactory the Interim Government and formed part of the agenda of the Better Business Forum and the Regulatory Reform Commission. Although it is difficult to point to any specific policy outcomes as a consequence of the ICA, the establishment of the Bangladesh Investment Climate Facility in IFC would almost certainly have contributed to considerable improvements in reducing bureaucracy and red tape had the Government not changed. In the Bangladesh context there is little doubt that the Bank was getting some traction on the investment climate agenda up until the end of 2008 Strategic relevance Satisfactory The ICAs are Bank products and are not intended to be and ownership Government owned. The first ICA was luckier than the second in terms of having a responsive government able to pick up on its recommendations. 48 Technical quality Satisfactory The ICA is clear and largely focused on diagnostics. It establishes a baseline for follow up analysis. Dissemination and Satisfactory On paper it is difficult to do it better and yet the substantial effort dialogue failed to lodge the document and analysis in the consciousness of the counterparts. Naturally, a substantial amount of time has passed since this dissemination took place. Overall assessment The primary achievement was to provide an evidence base for ―what everyone new‖ so that it was no longer just what the Bank was saying, but had some objective basis. In Bangladesh where skepticism is the order of the day, this is a quite notable achievement. Ratings Summary 2008 ICA Criterion Rating Comments Results Moderately The significant follow up to the first ICA was stillborn when the Unsatisfactory new government came in. Nothing has happened on this front and BICF has focused on capacity building rather than pushing to try to have an impact on policy. In the Bank‘s subsequent work it also appears to have downplayed investment climate issues in the dialogue and in the Development Support Credits (DSCs),. Nevertheless, there is some slow progress on the investment climate area and the ICA may have helped to keep this on track. Strategic Moderately While there was ownership by the interim government, the new relevance and Unsatisfactory government took a different line. There is little evidence that the ownership Bank considered this risk. This might have led to a decision to delay the start of the second ICA until a new Government was in place that could then take some ownership over the new exercise. Technical quality Moderately Although highly sophisticated technically, it was not as well Satisfactory adapted to the needs of getting clear messages across to an audience as the first ICA. Dissemination Satisfactory There was a thorough effort at disseminating the ICA, with and dialogue substantial press coverage and high-level attendance. Three years later, recollection of the ICA or of these events was limited. Part of the problem perhaps lies in the complex presentation and the lack of clear messages. Overall The overall assessment is rather negative , which reflects the assessment substantial effort and resources spent on the exercise, with low results and ownership. 49 Detailed Assessments: 2003 ICA Note: The tables in each section are not ratings sheets. They are tally sheets that allow the evaluator to tick off key characteristics of each AAA product. The number of checks in each column may be summed to help inform an overall rating for each product. A. Results Comments Results indicators Did the product have… …clear results / objectives defined at Yes. The concept note set out plans and provided detail on inception? the surveys. …clearly defined indicators defined at Yes. Most of the indicators are those used in the cross- inception? country core questionnaire of the enterprise surveys. …a clear strategy to achieve results (log- Yes. frame)? To what extent were the expected results and results-chain explained to client? Results achieved In the client country, were the findings used… …in government policy law or regulation? No, but the ICA contributed to the setting up of the Regulatory Reform Commission (RRC) and the Better Business Forum (BBF), and helped with the design and reform of regulations. The effort was not continued, however, under the new Government and there were few concrete results. …in design of spending or investment No. The ICA stressed the need for investment in energy programs? supply and made the powerful point that Bangladesh was losing 2 percent of GNP growth each year because of energy deficiencies. This became part of the public consciousness yet had little to no impact on the design of spending or investment programs. …to raise stakeholder awareness / capacity? Yes. The ICA helped to build some capacity to carry out the enterprise surveys. Other impact on capacity building was mostly indirect: through the Bangladesh Investment Climate Facility (BICF) to support Government efforts to improve the investment climate. See also the section on Dissemination and Sustained dialogue. …to build a coalition for change? Yes/NO. The main coalition for change in the investment climate was made up by the above mentioned establishment of the RRC and the BBF. These were influenced by the BICF, which –in turn—was influenced by the first ICA and DBIs. However, the RRC and BBF were later discontinued, with the arrival of the new government. …to influence the donor community? Yes. Most prominently, the ICA contributed to a decision by DFID and IFC to establish the BICF in 2006. 50 In the Bank, were the findings used in …Bank strategy formulation (CAS/PRSP)? YES. One of the pillars in the 2006 CAS (which was aligned with the government‘s Poverty Reduction Strategy) was dedicated to improving the investment climate, and reflected messages from the ICAs. As the main messages from the ICA were broadly known, it would be too much to say that the ICA shaped the CAS strategy. …the design of Bank lending products or SOMEWHAT. Interviewees point mostly to influence of the AAAs? ICA on the establishment of the BICF in 2006. If there were recorded results, is there YES/NO. Much of the momentum for investment climate evidence that they are likely to be reform that the ICA contributed to has dissipated. That said, sustainable? the BICF is still operational and building capacity. The ICA instrument can still play an important role when the political context puts greater weight on investment climate issues. Monitoring of impact Did the Bank or the client undertake an No clear results framework was set up, and no assessment of assessment of the product‘s impact? the impact took place. What intermediate outcomes or indicators The ICA provided useful validation and quantification of reflect the influences of the ICA? ―what everyone knew‖, and helped built some capacity to carry out enterprise surveys. Other capacity building efforts were addressed by the BICF. What results / outcomes are attributable to The first ICA influenced the decision to set up the BICF, and this ICA? further down the results chain, the BICF advised the installation of the Regulatory Reform Commission (RRC) and a Better Business Forum (BBF). Discussion of Results Results of the first ICA are rated Moderately Satisfactory. The first ICA innovated the enterprise sampling and survey instrument to Bangladesh and helped build capacity to carry out such surveys. It provided useful validation and quantification of ―what everyone knew‖, especially regarding energy as an obstacle to doing business. Together with other analytical work by the Bank such as the Doing Business Indicators, the ICA moved the discussion of investment climate issues to center stage in Bangladesh and this contributed to a decision by DFID and IFC to establish a Bangladesh Investment Climate Facility (BICF) in 2006 to support Government efforts to improve the investment climate. In turn, the BICF provided advice on the installation of the Regulatory Reform Commission (RRC) and a Better Business Forum (BBF), which formed the main public-private vehicles through which discussion on the investment climate took place. This generated considerable momentum, which proved not to be sustainable in the medium term. 51 B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. 26 Comments Strategic relevance Was the delivery of the output timely and on YES. The first ICA was prepared in 2001/02 and schedule? published in 2003, which may seem long but it was one of the first of its kind and required time to establish the first baseline. Was the issue identified as a development constraint in (note all that apply)… …the relevant CAS? YES. An early CAS (2000) identified private sector development as one of its four development priorities, and singled out infrastructure (especially power, telecommunications and transport) as one of the two major constraints to doing business (along with social and political uncertainty). …previous lending or nonlending work? TO SOME EXTENT. The Doing Business Indicators (DBIs) for Bangladesh indicated the need for Bangladesh to improve its investment climate and helped to create openness to the ICAs and the BICF. In addition to other themes, the ICA aimed to point at the obstacles that the poor electricity supply posed to the investment climate, which was not before covered in Bank AAA—in terms of allowing the business community to be heard (through the enterprise survey). In this sense the first ICA was important in raising consciousness. …policy dialogue with clients? YES. The Bangladesh Nationalist Party (BNP) that took office in 2001 and identified improving the investment climate as one of its focal points. In the policy dialogue, the ICA initiative clearly resonated with the government‘s focus. …donor coordination forums? YES. A limited number of donors are active in this area (DFID, ADB) and were all interested in this topic. Later on, DFID contributed to the financing of the associated IFC investment climate program that started in 2006. In cases where the dialogue is difficult, did the NO. In the area of improving power capacity, the Bank product, did the task find a solution? made little headway before or after the ICA despite the stress it puts on the issue. There was somewhat more progress on corruption and bureaucracy, but it has been very slow. 26 See IEG (2008). Using knowledge to improve development effectiveness: An evaluation of World Bank economic and sector work study. Paragraph 5.21 through 5.29; and Development Results in Middle-Income Countries: An Evaluation of the World Bank‘s Support study page 37. 52 Comments Ownership Is the product part of an overall program of YES. As mentioned above, the initiative for the ICA AAA to which the authorities have contributed neatly fit the government‘s focus and policy agenda. The or agreed? ICA was not identified as a deliverable in the 2000 CAS, but mentioned as a recently delivered ESW in the 2003 CAS Progress Report. That said, the ICA was initiated in coordination with the government. Did the client request or commission the NO. specific product? Did the client cover some or all of the costs? NO. Did key decision makers collaborate with, YES. There was some discussion ex ante of the purpose discuss or provide feedback on the product? and design of the activity. Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? YES. The institute involved in the surveys was active in the planning phase. …analyze the results and write the report? NO. …provide peer review or comments on the NO. They were invited to the workshops but did not take draft report? a major part in the discussion. …organize workshops or discussions about the NO. findings? Discussion of Strategic relevance and ownership Strategic relevance and ownership is rated Satisfactory for the first ICA. A key motivation for conducting the 2003 Bangladesh ICA was to provide concrete evidence for the magnitude of problems in the supply of electricity which had long inhibited investment and growth in the economy. This contributed to the strategic relevance of the ICA. In addition, ownership of the ICA was assured as the Bangladesh Nationalist Party (BNP) that took office in 2001 had identified improving the investment climate as one of its focal points. Results of the enterprise survey indeed painted a disturbing picture of the effects of the poor power supply. Its poor quality and reliability forced most private firms to incur substantial costs of running in-house private generators. In addition, the ICA analysis showed that inadequate power supply depressed economic growth by about two percentage points each year. The findings resonated deeply with the general public and the incumbent administration. The ICA helped the WBG advance its country dialogue and encouraged the government to initiate regulatory reforms and preparation for new investments in the power sector. C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. make use YES. of current and relevant knowledge available)? 53 …cite relevant examples of practice or research YES. There is a great deal of coverage of comparator from other countries? findings from Asian and other countries. …discuss the specific institutional and policy YES. It is in the ICA, but lightly handled given that context for the issue? there is already a very complex agenda to be covered. …collect and analyze existing local data? YES. Undoubtedly the ICA is the most significant data set on enterprises in Bangladesh available, especially given the weakness of the industrial census. …generate new evidence? YES. There is unanimity among the interlocutors that there were ―no surprises‖. That said, it brought to the fore the views of the business community which added value to the policy dialogue, especially on energy issues. …include recommendations? YES. The recommendations are a reflection of other Bank work and broadly consistent with the ICA rather than arising out of the ICA analysis. Is the policy advice consistent with the preservation of social order? Do the recommendations include specific actions YES. The authors attempted to provide specific and to be taken by specific actors? operational recommendations. Was the task team staffed with the necessary YES. The teams were strongly staff on FPD issues and expertise? technical survey issues. It is difficult to cover all the sectoral issues that relate to the investment climate and for this teams must rely on coordination with their sector colleagues. Did the product receive adequate managerial The Country Director was extremely interested in the attention? ICAs and involved in the concept and review stage. Did the product receive a sufficient budget? YES. The ICA was initially budgeted at $50,000 (which seems unrealistically low), and the actual final costs are recorded (in the Bank‘s Business Warehouse) at $161,000. It is not clear if these costs reflect both staff time and the costs for the enterprise survey. Review of contents Was the draft peer-reviewed by appropriate YES. The ICA required that some of the review be experts? done by fellow practitioners who are part of the thematic/practice group. This was the case here. Were the peer-review comments taken into YES. account? Was the feedback from the client incorporated? YES. Discussion of Technical quality Technical Quality of the first ICA is rated Satisfactory. It was one of the pilot exercises and the enterprise survey helped set a baseline on the basis of which a second ICA and enterprise survey could be conducted. It set realistic goals and achieved them adequately with a very clear and prioritized presentation. 54 D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local language? No. But English is widely spoken. …made available on a website? No/Yes. The ICA was one of the early pilot exercises and is not available on the Bank‘s external website. …discussed with senior policy-makers? YES. The ICA was discussed at fora with ministers and senior officials present. …presented at a workshop, conference or YES. A major effort at dissemination was made for the ICA. on the web? The conference on the first ICA was regarded as a model of its kind. …covered in the media? YES. Extensive coverage in the press. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? YES/NO. The first ICA had a considerable impact on the policy dialogue, along with other ESW such as the DBIs. …discussions/contact with stakeholders? YES. There was extensive contact with the private sector on the first ICA as a consequence of the establishment of various public/private partnerships. In the medium term, this was not sustained during the new Government and the second ICA. …lending and non-lending products? YES. The main vehicle was the Bangladesh Investment Climate Facility (BICF), which was jointly initiated by IFC and DfID. One of the pillars in the 2006 was dedicated to improving the investment climate, and reflected messages from the ICAs. Discussion of Dissemination and sustained dialogue Dissemination and sustained dialogue is rated Satisfactory. The Bank organized major events and helped to fuel the discussion on the investment climate. In many ways, the Bank did as much as it could. That said, the participants‘ recollection of the launch events is vague, and the dialogue on investment climate issues was not sustained. The first ICA was launched at a major event and included high level representatives, for instance on the ministerial level, as well as guest speakers from India, Pakistan, and Sri Lanka. Working groups were organized on power, telecommunications, ports, business regulation, and accountability of the civil service. The message that Bangladesh was losing two percentage points of GDP growth each year because of inadequacies and interruptions in power supply was especially resonant. The formation of BICF provided an effective follow up instrument for the ICA. However, over the medium term, this did not prove sustainable—see the questionnaire for the second Bangladesh ICA. 55 Detailed Assessments: 2008 ICA Note: The tables in each section are not ratings sheets. They are tally sheets that allow the evaluator to tick off key characteristics of each AAA product. The number of checks in each column may be summed to help inform an overall rating for each product. A. Results Comments Results indicators Did the product have… …clear results / objectives defined at Yes. The concept note set out plans and provided detail on the inception? surveys. It was discussed with the Government. …clearly defined indicators defined at Yes. This is the second ICA in the country. The standard inception? indicators are already known. …a clear strategy to achieve results (log- Yes. frame)? To what extent were the expected results To some extent. There was some discussion of the coverage of and results-chain explained to client? the second ICA with the Interim Government which led to a broadening of the coverage. Results achieved In the client country, were the findings used… …in government policy law or regulation? No. Through the Bangladesh Investment Climate Facility (BICF), the ICA contributed to the setting up of the Regulatory Reform Commission (RRC) and the Better Business Forum (BBF). This took place in the period when the caretaker government was in place. But the effort was not continued under the new Government and there were few concrete results. …in design of spending or investment No. programs? …to raise stakeholder awareness / Yes. The ICA helped to build some capacity to carry out the capacity? enterprise surveys. Other impact on capacity building was mostly indirect: through the BICF (which has been operational since 2006) to support Government efforts to improve the investment climate. See also the section on Dissemination and Sustained dialogue. …to build a coalition for change? Initially, yes. The main coalition for change in the investment climate was made up by the RRC and the BBF. These were influenced by the BICF, which –in turn—was influenced by the first ICA and DBIs. However, the RRC and BBF were discontinued with the arrival of the new government. …to influence the donor community? Hardly. The joint IFC-DfID BICF did continue through the time when the second ICA was conducted. In the Bank, were the findings used in …Bank strategy formulation (CAS/PRSP)? Yes. One of the pillars of the 2010 CAS (which was aligned with the government‘s Poverty Reduction Strategy) was dedicated to improving the investment climate, and reflected messages from the ICA. As the main messages from the ICA 56 were broadly known, it would be too much to say that the ICA shaped the CAS strategy. …the design of Bank lending products or Somewhat. Interviewees point mostly to influence of the ICA AAAs? on the Development Support Credits (DSCs), but some were of the view that the Bank could have done more if it had stuck to its guns on getting progress on the Investment Climate as part of the results framework. If there were recorded results, is there No. Much of the momentum for investment climate reform that evidence that they are likely to be the ICA contributed to has dissipated. That said, the BICF sustainable? (influenced by the first ICA) is still operational and building capacity. The ICA instrument can still play an important role when the political context puts greater weight on investment climate issues. Monitoring of impact Did the Bank or the client undertake an No. In a sense the second ICA should have provided an assessment of the product‘s impact? assessment of the impact of the first, but it has so much data that it is hard to get a clear assessment of whether the investment climate improved in the intervening period. This is something that should be thought through. The DBIs with their clear and simple message are much more powerful in this regard. What intermediate outcomes or indicators Like the first ICA, the second one provided useful validation reflect the influences of the ICA? and quantification of ―what everyone knew‖ (certainly regarding the energy sector), and helped built some capacity to carry out enterprise surveys. Other capacity building efforts were continued through the BICF. What results / outcomes are attributable to The first ICA influenced the decision to set up the BICF, and this ICA? further down the results chain, the BICF advised the installation of the Regulatory Reform Commission (RRC) and a Better Business Forum (BBF). The second ICA added to this momentum, which later faded. Discussion of Results Results of the second ICA are rated Moderately Unsatisfactory. Initially, the ICA continued the momentum that the first ICA contributed to. The ICA team provided a set of recommendations to the working groups that were formed by BBF. However, timing proved to be unfortunate, as the momentum dissipated with the arrival of the new government. The agenda for investment climate reform was concentrated at the ministerial level, and the RRC‘s and BBF‘s mandate were not extended. At this point, the Bank‘s efforts on the investment climate are mostly channeled through the ongoing BICF, while some investment climate topics were addressed through Development Support Credits (DSCs). There is little to show in terms of concrete follow up on the investment climate agenda. While a number of interviewees indicated that they viewed as a major contribution that the ICA provided an evidence basis for ‗what everyone knew‘, at some point this needs to be translated into results on the ground. This said it does not argue for stopping the surveys, but perhaps putting less effort into glossy reports and more into simple well publicized benchmarking of progress over time. 57 B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. 27 Comments Strategic relevance Was the delivery of the output timely and on No. The second ICA was extremely ambitious and this schedule? required extra time, which meant that some of the Bank staff and Government counterparts had moved on between start and finish. Also see the write-up below. Was the issue identified as a development constraint in (note all that apply)… …the relevant CAS? Yes. At the time of the 2006 CAS, the topic of improving the investment climate was already identified as a priority in the Bank‘s strategies. . …previous lending or nonlending work? Yes. The focus on certain constraints was tailored on the basis of overlap with the previous ICA, while a wider scope was used. …policy dialogue with clients? Yes. The caretaker government took a clear interest in improving the investment climate. …donor coordination forums? Yes. A limited number of donors are active in this area (DFID, ADB) and were all interested in this topic. DFID contributed to the financing of the associated ongoing IFC investment climate program. In cases where the dialogue is difficult, did the Not sufficiently. At the request of the country director, the product, did the task find a solution? ICA team went beyond the diagnostics and developed a set of policy recommendations for government and donor follow-up. That said, these were not the consequence of serious sector analysis, but tended to reflect off-the-shelf solutions to the problems identified. Ownership Is the product part of an overall program of Yes. The second ICA was identified as a delivery in the AAA to which the authorities have contributed 2006 CAS, which committed one of its three pillars to or agreed? improving the investment climate. Did the client request or commission the specific To some extent. The interim government suggested product? expanding coverage to rural areas and informal sector. Did the client cover some or all of the costs? No. Did key decision makers collaborate with, Yes. There was some discussion ex ante of the purpose and discuss or provide feedback on the product? design of the activity. 27 See IEG (2008). Using knowledge to improve development effectiveness: An evaluation of World Bank economic and sector work study. Paragraph 5.21 through 5.29; and Development Results in Middle-Income Countries: An Evaluation of the World Bank‘s Support study page 37 58 Comments Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? Yes. The institute involved in the surveys were active in the planning phase. …analyze the results and write the report? No. …provide peer review or comments on the No. They were invited to the workshops but did not take a draft report? major part in the discussion. …organize workshops or discussions about the No. findings? Discussion of Strategic relevance and ownership Strategic relevance and ownership is rated Moderately Unsatisfactory for the second ICA. Had this review been done in 2008 during the interim government the rating for relevance and ownership would have been Satisfactory. The Interim Government had internalized the discussion of the ICA and put in place an excellent institutional framework in the Better Business Forum that could serve as a counterpart for the ICA and provide real ownership and feedback on its recommendations. Two years later, the reality is a Government that has distanced itself as much as possible from the Interim Government and has allowed the institutions it created to disappear and has shown little interest in the ICA recommendations or follow up. The issues are first, could the Bank have done more to build ownership of the second ICA and the process as a whole, and second whether there is ownership within the Bank of the recommendations and the follow up in the Bank‘s own analytic work and particularly the Bank‘s development policy lending. On the first while the Bank could not have foreseen that the Prime Minister would be so antipathetic to the investment climate agenda because of its links with the Interim Government, it could have delayed the second ICA until a new Government was in place and tried to ensure that it would be seen as a new exercise and a joint product. On the second issue, in the view of most interlocutors the relevance and impact of the ICAs in Bangladesh will depend on the willingness of the Bank to put its money where its mouth is on investment climate issues. Some of those interviewed felt that this had not happened as yet and that the Bank in some sense was underplaying its hand in Bangladesh on this set of issues. C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. make use Yes. of current and relevant knowledge available)? …cite relevant examples of practice or research Yes. There is a great deal of coverage of comparator from other countries? findings from Asian and other countries. …discuss the specific institutional and policy Yes. It is in the report, but lightly handled given that context for the issue? there is already a very complex agenda to be covered. …collect and analyze existing local data? Yes. Undoubtedly the ICA is the most significant data set on enterprises in Bangladesh available, especially given the weakness of the industrial census. …generate new evidence? Yes. There is unanimity among the interlocutors that there were ―no surprises‖. There was interest in the greater emphasis attached to land and labor constraints in the second survey than the first. 59 …include recommendations? Yes. But, the recommendations are a reflection of other Bank work and broadly consistent with the ICA rather than arising out of the ICA analysis. Is the policy advice consistent with the preservation of social order? Do the recommendations include specific actions Yes. The authors attempted to provide specific and to be taken by specific actors? operational recommendations Was the task team staffed with the necessary Yes. The teams were strongly staff on FPD issues and expertise? technical survey issues. It is difficult to cover all the sectoral issues that relate to the investment climate and for this teams must rely on coordination with their sector colleagues. Did the product receive adequate managerial Yes. The CDs were extremely interested in the ICAs and attention? involved in the concept and review stage. Did the product receive a sufficient budget? The original budget of $50,000 for the second ICA was simply a place marker. The actual final costs (As recorded in the Bank‘s Business Warehouse) were$500,000. It is not clear if these costs reflect both staff time and the costs for the enterprise survey. Review of contents Was the draft peer-reviewed by appropriate Yes. The ICA required that some of the review be done experts? by fellow practitioners who are part of the thematic/practice group. This was the case here. Were the peer-review comments taken into All the peer reviewers commented on the complexity of account? the presentation of the second ICA. There is little evidence that much was done to make it simpler, clearer and more prioritized. Was the feedback from the client incorporated? Yes. Discussion of Technical quality Technical Quality of the second ICA is rated Moderately Satisfactory. As compared with the first ICA, it was far more ambitious incorporating a huge module on the rural sector. The scope of the questionnaire required multiple visits and took 3 to 4 hours of senior executives‘ time to complete. The analysis is sophisticated and of high quality, but the report is so laden with comparative data – cross-country, inter-temporal, spatial, sub- sectoral – that it becomes almost impossible to wade through one dense paragraph after another. The initial report was largely diagnostic in nature, but the CD insisted that it include operational recommendations. The report drew on the considerable body of existing sector analysis for this purpose. The recommendations are sensible, but do not always therefore come easily out of the analysis in the document. 60 D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local language? No. But English is widely spoken. …made available on a website? Yes. In addition the data set is also available. …discussed with senior policy-makers? Yes. Both ICAs were discussed at fora with ministers and senior officials present. …presented at a workshop, conference or on Yes. A major effort at dissemination was made for both. the web? The conference on the first ICA was regarded as a model of its kind. …covered in the media? Yes. Extensive coverage in the press. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? Hardly. By the time the second ICA was produced the Government showed little interest in IC issues and this has not been the subject of policy dialogue. …discussions/contact with stakeholders? Hardly. There was extensive contact with the private sector on the first ICA as a consequence of the establishment of various public/private partnerships. This was not sustained during the new Government and the second ICA. …lending and non-lending products? To some extent. Interviewees point mostly to influence of the ICA on the Development Support Credits (DSCs) though some felt that the Bank could have done more to push these issues through the DSCs. One of the pillars in the 2010 CAS was dedicated to improving the investment climate, and reflected messages from the ICA. Discussion of Dissemination and sustained dialogue Dissemination and sustained dialogue is rated Satisfactory. The Bank organized major events and helped to fuel the discussion on the investment climate. In many ways, the Bank did as much as it could. That said, the participants‘ recollection of the launch events is vague, and the dialogue on investment climate issues was not sustained. The strategy behind the dissemination of the second ICA was elaborate and well thought through. Aiming for best practice, the ICA team consulted with the external affairs unit and used the dissemination of the Turkey ICA as a benchmark. Launch events were organized in Dhaka and Chittagong as collaborative efforts with the Board of Investment and Chamber of Commerce, respectively. These were well attended, and received considerable press coverage. However, the Bank organizes so many launch and dissemination events in a country such as Bangladesh that few of the participants in these workshops can actually remember participating in them. In addition, the sustainability of the ensuing policy dialogue proved to be too fragile as the new government disbanded the Regulatory Reform Commission (RRC) and the Better Business Forum (BBF), which formed the main public-private vehicles through which IC discussion took place 61 ANNEX 2: EGYPT Performance Assessment Report on Investment Climate Assessments Egypt Investment Climate Assessment - 2004 Vice President Christiaan J. Poortman Country Director Hossein Razavi Sector Director Emmanuel Mbi Sector Manager Zoubida Allaoua Task Leader Najy Benhassine Project ID/ Type/ Report No./Cost P089111/Client Request / 36675/$334,000 Number of: Volumes/ Pages 1 Volume/ 142 pages Languages English Document Type Investment Climate Assessment Topics Private Sector Development Unit Financial and Private Sector Development (MNSIF) Egypt Investment Climate Assessment - 2009 Vice President Daniela Gressani Country Director Emmanuel Mbi Sector Director Ritva Reinikka Sector Manager Zoubida Allaoua Task Leader Andrew H. W. Stone Project ID/ Type/Cost P108292/Client Request/$301,000 Number of: Volumes/ Pages 1 volume / 152 pages Languages English Document Type Investment Climate Assessment Topics Private Sector Development Unit Social and Economic Development Group - MNSED Background Three assessments have been undertaken, in 2004 (finalized in 2005), 2006 (an update with a report produced in 2007) and 2009. This is an unusually high frequency and is a consequence of the interest of the then Minister of Investments (Dr. Mohieldin) in using the ICA as a tool both to set the reform agenda, and perhaps more importantly as an independent assessment of progress in implementation of the reform. The Minister and the Ministry were the principal audience of the ICA and reviewed and discussed the document carefully, but in line with Government policy for ESW in Egypt, particularly for documents dealing with sensitive issues such as the prevalence of corruption, the ICAs were not made public and dissemination among the business community and academia was limited to powerpoint presentations. Overall Assessment The Egypt ICA process has been very effective. For the 2004 ICA, the relevance of the ICA is rated Highly Satisfactory and the results and quality of the ICA were satisfactory, the failure to disseminate the document widely is rated Moderately Unsatisfactory. The 2009 ICA is rated Satisfactory in all dimensions other than Dialogue and Dissemination which is rated Moderately Satisfactory. While the limited dialogue and dissemination were largely beyond the Bank‘s control, it is increasingly apparent that lack of transparency has cost the Government dearly, and this reflects on the Bank. There are serious issues for the institution of whether it should agree up front to the Government vetoing the publication of a document such as this. While the data are available on the Bank‘s web-site, the Bank does not appear to have made much effort to provide Egyptian researchers with links to the data, nor has it publicized the availability of an important data set. 62 Lessons As indicated, the key lessons relate to the importance of reaching an understanding up-front on the public availability of the findings of the ICA. This said, there are a number of highly positive lessons that emerge from the Egypt experience. First, there is the fact that the Minister of Investments was a powerful champion and source of demand for the ICAs and the presence of a well-placed champion was largely accountable for the considerable impact of the analysis. Second, the experience demonstrates that what matters is not ownership of the preparation process – in fact it was in large part the arms length nature of the ICA that gave it credibility in the eyes of the Government. The importance of ownership rests in the willingness to take the analysis seriously and use it as a basis for policy. Third, the ICAs were used as a monitoring tool with a two-yearly frequency. This high frequency probably only makes sense if the Government is actually pursuing an active reform agenda. In this case there was a striking improvement in perceptions about the level of taxes and the honesty of the tax collection process which showed clearly in the data and was used by the Minister to maintain the interest of his cabinet colleagues in the reform process. Fourth in Egypt as in Bangladesh, the expansion of the 2009 ICA to small and informal enterprises, as well as to upper Egypt, produced limited though useful information on a one time basis, and suggests that while special extensions of the data collection may be appropriate from time to time, these should not necessarily be routinely undertaken. Ratings Summary 2004 ICA Criterion Rating Comments Results Highly Satisfactory The Ministry of Investments used the ICA as helping to set the reform agenda and as a monitoring tool, but because it was closely held it did not contribute to a wider public dialogue that could promote longer-term impact. Strategic relevance Highly Satisfactory The timing was perfect given the creation of a new and ownership ministry with the mandate for promoting investment. The Minister turned out to be a champion of ICAs. Technical quality Satisfactory Very solid and well crafted reports. They make effective use of other Bank ESW and are clear and straightforward. Dissemination and Moderately While the context is well understood and largely explains dialogue Unsatisfactory the lack of dissemination, the rating needs to be interpreted as a comment on the Government‘s lack of transparency and the neglect of making the analysis part of the public consciousness in Egypt in the longer term. It is notable that a similar OECD investment climate assessment seems to have been much more widely disseminated in Egypt. Overall assessment The short to medium term impacts were very positive, but the longer-term impact is diminished by the fact that the lack of publication meant that the ICA could not contribute to a much needed public discussion of these issues. 63 Ratings Summary 2009 ICA Criterion Rating Comments Results Highly Satisfactory Although there was a positive impact the authorities seem to have taken the view that the key changes were in process and the emphasis was on consolidating the achievements. It was still valuable however in maintaining cabinet commitment to the reforms. Strategic relevance Satisfactory Remained relevant and was particularly useful as a and ownership benchmarking tool in demonstrating areas of progress and also in its examination of special issues relating to the informal sector and Upper Egypt. Technical quality Satisfactory A very well crafted and solid document which was well prioritized and by the standards of many other ICAs relatively easy to read. Dissemination and Moderately There was considerably more effort to disseminate the Dialogue Unsatisfactory 2009 ICA than in the case of the 2004 ICA, but still based largely on power point presentations. The full report was not made generally available. Overall assessment Despite the fact that this was the third survey, the ICA remained useful to the Ministry of Investments in its internal dealings with other parts of the Government if not quite as powerful in terms of policy reform as the previous ones. 64 Detailed Assessments: 2004 ICA Note: The tables in each section are not ratings sheets. They are tally sheets that allow the evaluator to tick off key characteristics of each AAA product. The number of ticks in each column may be summed to help inform an overall rating for each product. A. Results Comments Results indicators Did the product have… …clear results / objectives defined at inception? Yes. This went beyond the standard objectives of the ICA given very clear indications by the Minister of what he wanted and what he expected to get from the ICA. …clearly defined indicators defined at Yes. The indicators are standard, but were carefully inception? discussed with the authorities. …a clear strategy to achieve results (log- The ICA is largely a diagnostic tool and, although there are frame)? recommendations, these tend to be at a level of generality that requires follow up work for implementation. To what extent were the expected results and the YES. In this case it was the client that was setting up the results-chain explained to the client? results chain and in fact using the ICA to check whether the results in terms of policy reforms were achieving their purpose. Results achieved In the client country, were the findings used… …in government policy law or regulation? YES. The findings were used in a number of policy and regulatory areas. For example, the finding on the large number of inspection visits to firms was particularly influential. …in design of spending or investment YES. Government investment in infrastructure and programs? economic zones, while not directly attributable, was in line with recommendations. …to raise stakeholder awareness / capacity? Insofar as the stakeholder group was the Government, there was capacity building provided. Also of course the survey team received training. Although the broader business community was not directly assisted, there was IFC follow up training for the Alexandria Business Association. …to build a coalition for change? NO. A major gap reflecting the Government‘s concern for absolute control. …to influence the donor community? NO. Some early involvement with USAID but overall fairly limited inter-action or discussion with donors. In the Bank, were the findings of the product used in …Bank strategy formulation (CAS/PRSP)? Yes. An important input into the Bank‘s strategy (CAS 2006-09) and the focus of its program. …the design of Bank lending products or Yes, including the Bank‘s financial sector DPLs, AAAs? infrastructure support, incl. a power project in 2006, and a series of policy notes took off from the ICA. 65 Comments If there were recorded results, is there evidence Difficult moment to assess, but the institutional framework that they are likely to be sustainable? is in place. Monitoring of impact Did the Bank or the client undertake an In a sense the ICA is self-monitoring – in this case the assessment of the product‘s impact? monitoring aspect was unusually important. What intermediate outcomes or indicators reflect Creation of new economic zones, policy initiatives in a the influences of the ICA? number of regulatory areas and also in reducing the number of inspections, increased emphasis on infrastructure. What results / outcomes are attributable to this Probably too early for attribution of outcomes. ICA? Discussion of Results Results of 2004 ICA are rated Highly Satisfactory. The timing of the first ICA was ideal in that Dr. Mohieldin was appointed as Minister of Investments and was looking for an instrument to bring other ministries into the reform agenda and quickly saw the potential value of the ICA in this regard. The ICA series was also valued as an instrument for benchmarking the progress that the Ministry was achieving, and also served to prioritize the Ministry‘s efforts. On the Bank side the work that was undertaken contributed to the financial sector work as well as the 4th DPL – arguably the DPL and the decision to proceed with policy lending grew out of the ICA. The one issue where there was not much mileage in the short term was that of access to land where the interests of the military prevented significant policy action. The Government found the information on the frequency of inspections particularly useful information. This enabled the Ministry of Investments to take a stand on reducing these visits which in their view simply produce time-wasting and lead to informal payments. In the words of one staff member, the ICA gave the Bank a ‗toe-hold‘ on policies in Egypt, where it had had none before. B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. 28 Comments Strategic relevance Was the delivery of the output timely and on YES. This was largely a consequence of the Minister‘s desire schedule? for a monitoring framework. In fact the Ministry would like to maintain a regular two year frequency. Was the issue identified as a development Yes. constraint in (note all that apply)… …the relevant CAS? Yes. Egypt‘s relative lack of success in attracting FDI and the desperate need to generate employment were central features of the 2001 CAS (covering FY02-04). …previous lending or nonlending work? Mainly ESW which had noted this as an important constraint. …policy dialogue with clients? YES. The Bank had stressed this as an issue in its policy dialogue. 28 See IEG (2008). Using knowledge to improve development effectiveness: An evaluation of World Bank economic and sector work study. Paragraph 5.21 through 5.29; and Development Results in Middle-Income Countries: An Evaluation of the World Bank‘s Support study page 37 66 Comments …donor coordination forums? NO. Interest mainly on the part of USAID which was planning a major initiative on regulatory reform. In cases where the dialogue is difficult, did In the area of corruption, the proxy variable of the number of the task find a solution? inspection visits constituted a real breakthrough to addressing the issue in a less confrontational way. On the difficulties of increased transparency, however, the ICA made little headway. Ownership Is the product part of an overall program of YES. Very strong interest in the ICA – perhaps more than in AAA to which the authorities have other Bank AAA. contributed or agreed? Did the client request or commission the YES. Both the frequency and the coverage were determined by specific product? the client‘s interests. Did the client cover some or all of the costs? NO. The issue did not arise. Did key decision makers collaborate with, The Minister dominated the feedback. For the rest it was a discuss or provide feedback on the product? question of internalizing the ICA findings rather than any questioning or pushback. Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? YES. The American University of Cairo (AUC) was the contractor for the survey and made minor contributions to the design. …analyze the results or write the report? No. …provide peer review or comments on the NO. Although invited to the meetings AUC did not contribute draft report? much. …organize workshops or discussions about The Ministry organized the discussion. the findings? Discussion of Strategic relevance and ownership Strategic relevance and ownership of the 2004 ICA are rated Highly Satisfactory because of the timing and the presence of the new Minister as a champion for the use of the analysis. The Minister saw the first ICA as providing a baseline on the regulatory environment and the subsequent ones as providing a monitoring tool. It was largely at his urging that the ICA was undertaken at two or three year intervals. The Minister himself went through the questionnaire before it was approved, and the ministry sent a letter to the firms selected asking/instructing them to cooperate with the survey team from the American University of Cairo. The Bank provided training along with the ICA both to the institute that prepared the survey, but also to the Ministry which lacked staff with a suitable technical background. Training was provided on how to read the findings. 67 C. Technical quality Comments Quality of content Did the product… . .. include appropriate knowledge, i.e., make YES. Fully used the existing ICA methodology. use of current relevant knowledge? …cite relevant examples of practice or research YES. There are the usual cross country comparisons, though from other countries? these were of less interest to the authorities than the inter- temporal comparisons within Egypt. …discuss the specific institutional and policy YES. A very important part of the discussion in the ICAs context for the issue? particularly in regard to regulation. …collect and analyze existing local data? …generate new evidence? YES. The ICA represented a new data set. There are other enterprise censuses and surveys, but none geared to investor perceptions. …include recommendations? YES. The recommendations were useful in agenda setting, but required more specific follow up ESW to derive actionable approaches. Is the policy advice consistent with the YES. The authorities clearly thought that the analysis and preservation of social order? especially the treatment of corruption issues was potentially dangerous to the regime and should be closely held. On the other hand with hindsight had there been more openness about the presence of corruption and an associated willingness to take on the issue perhaps this might have contributed to a more orderly reform process. Do the recommendations include specific Yes, but they needed follow up to provide practical actions to be taken by specific actors? proposals for implementation. The Egyptians did not necessarily see this as the role of the Bank, however. The diagnostics were what they sought. Given their own knowledge of the political economy, they could then work out the best way to proceed on any given issue. Was the task team staffed with the necessary Yes. In addition, the task team also drew on Bank sectoral expertise? analysis and expertise. Did the product receive adequate managerial YES. The Country Director was fully engaged in the attention? discussion of both ICAs and saw this as a key ESW product given the Minister‘s interest. Review of contents Was the draft peer-reviewed by appropriate YES. The peer review group includes peer reviewers drawn experts? from a panel of ICA practitioners. Were the peer-review comments taken into Yes. account? Was the feedback from the client incorporated? Yes, particularly the views of the Minister. 68 Discussion of Technical quality Technical Quality of the ICA is rated Satisfactory. It was an important diagnostic tool, but although it goes beyond diagnosis, it needed follow up in specific areas to provide real analysis for policy reforms. Thus the Egypt ICA was followed up by a much more detailed report on land policy; a FIAS review of economic zones; a huge USAID project on regulatory reform; and work by IFC on corporatization of state enterprises. Many of these follow up documents use information drawn from the ICA. This ICA is very well presented, readable with well-chosen comparisons. It was evident from discussions in Cairo that there was limited interest in the international comparisons – a contrast with the DBIs. The real concern here was to track the improvements in perceptions over time. Given the good technical quality it is even more disappointing that the reports have not reached a wider audience. D. Dialogue and Dissemination Comments Dissemination Was the product… …made available in the local language? No …made available on a website? No. Survey results are available on the web. But the report is not. …discussed with senior policy-makers? Yes. Although the only direct discussion was with the Minister, he in turn used the ICA in discussions with other ministers. …presented at a workshop, conference Yes. Powerpoint presentations were made in workshops and or on the web? conferences. …covered in the media? Yes. There was substantial press coverage of the launch of the ICAs, but less of the findings. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? Yes. The Ministry of Investments and the Ministry of Finance became the Bank‘s key counterparts. The ICA provided a basis for ongoing dialogue. …discussions/contact with Yes. The TTL for the 2004 ICA moved to other assignments and stakeholders? there was little effort to reach out to other audiences …lending and non-lending products? Yes. The ICA contributed to the series of financial sector DPLs and a number of policy notes. Discussion of Dissemination and sustained dialogue Dissemination and sustained dialogue is rated Moderately Unsatisfactory. The ICA was only disseminated in Cairo. A one-day event was held. The dissemination was by powerpoint and done before the report was finalized. There was no dissemination of the final report and it was clear that the Government was not keen on the report being made public. This has to be understood in the Egyptian context where the Government is often so reluctant to release information that in some cases provides information to selected Bank staff on condition that other Bank staff are not to have access to that information. In the words of the Government the first ICA was not made public because it contained ―some value judgements that did not match the status quo e.g. that judges are not well paid and that is why judgements are not enforced. There were no supporting arguments for this.‖ The ICA is however widely shared within the Government including the PM. They agree on the further processing of the ICA. No formal comments on the document are provided by the Government. 69 Detailed Assessments: 2009 ICA Note: The tables in each section are not ratings sheets. They are tally sheets that allow the evaluator to tick off key characteristics of each AAA product. The number of ticks in each column may be summed to help inform an overall rating for each product. A. Results Comments Results indicators Did the product have… …clear results / objectives defined at Yes. This was the second ICA in Egypt. Both the Government inception? and task team were already familiar with the process. …clearly defined indicators defined at Yes. The indicators are standard, but were carefully discussed inception? with the authorities. …a clear strategy to achieve results (log- Not applicable. The ICA is largely a diagnostic tool and, frame)? though there are recommendations, they tend to be of a generality that requires follow up work for implementation. To what extent were the expected results and Yes. In this case it was the client that was setting up the results the results-chain explained to the client? chain and in fact using the ICA to identify whether the results in terms of policy reforms were achieving their purpose. Results achieved In the client country, were the findings used… …in government policy law or regulation? Yes. There was use made of the findings in a number of policy and regulatory areas. …in design of spending or investment Yes. Government investment in infrastructure and economic programs? zones while not directly attributable was in line with recommendations. …to raise stakeholder awareness / capacity? Insofar as the stakeholder group was the Government, there was capacity building provided. Also of course the survey team received training though less was needed than for the earlier ICA. Although the broader business community was not directly assisted, there is IFC follow up training for the Alexandria Business Association. …to build a coalition for change? A major gap reflecting the Government‘s concern for absolute control. …to influence the donor community? The presence of the TTL in the country office has contributed to a more extensive inter-action with other donors than was the case for the earlier ICA. In the Bank, were the findings of the product used in …Bank strategy formulation (CAS/PRSP)? An important input into the Bank‘s strategy and the focus of its program. 70 Comments …the design of Bank lending products or The financial sector DPL and support for micro and small scale AAAs? enterprises, as well as a series of ESW products took off from the ICA. If there were recorded results, is there Obviously a difficult moment to assess this, but the evidence that they are likely to be institutional framework is in place. sustainable? Monitoring of impact Did the Bank or the client undertake an In a sense the ICA is self-monitoring – in this case the assessment of the product‘s impact? monitoring aspect was unusually important. What intermediate outcomes or indicators Creation of new economic zones, policy initiatives in a number reflect the influences of the ICA? of regulatory areas, also in reducing the number of inspections and increased emphasis on infrastructure. What results / outcomes are attributable to Probably too early for attribution of outcomes to the 2009 ICA, this ICA? but the ICA process as a whole clearly contributed to the momentum for change on the investment climate and the choice of priorities. Discussion of Results The rating of Satisfactory for the second (2009) ICA, reflects the fact that it contributed to the momentum for change on the investment climate and the choice of priorities. The perception in the Ministry of Investments was that in the areas over which they had direct control, the measures that needed to be taken were already in place. In other areas which depended on action by other Ministries (other than Finance which was fully on board with the reform agenda) progress was slower and although the ICA helped in the internal dialogue, because it was not made publicly available it could not be used to as an instrument for lobbying by the media and the private sector. B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. 29 Comments Strategic relevance Was the delivery of the output Yes, this was largely a consequence of the Minister‘s desire for a timely and on schedule? monitoring framework. In fact the Ministry would like to maintain a regular two year frequency. Was the issue identified as a development constraint in … …the relevant CAS? Egypt‘s relative lack of success in attracting FDI and the desperate need to generate employment were central features of the CAS. …previous lending or nonlending Mainly ESW which had noted this as an important constraint. work? 29 See IEG (2008). Using knowledge to improve development effectiveness: An evaluation of World Bank economic and sector work study. Paragraph 5.21 through 5.29; and Development Results in Middle-Income Countries: An Evaluation of the World Bank‘s Support study page 37 71 Comments …policy dialogue with clients? Yes, the Bank had stressed this as an issue in its dialogue. …donor coordination forums? Interest mainly on the part of USAID which was undertaking a major initiative on regulatory reform. In cases where the dialogue is In the area of corruption, the proxy variable of the number of inspection difficult, did the product, did the task visits constituted a real breakthrough to addressing the issue in a less find a solution? confrontational way. On the difficulties of increased transparency however, the ICA made little headway with the client. Ownership Is the product part of an overall Very strong interest in the ICA – perhaps more than in other Bank program of AAA to which the AAA. authorities have contributed or agreed? Did the client request or commission Both the frequency and the coverage were determined by the client‘s the specific product? interests. Did the client cover some or all of The issue did not arise. the costs? Did key decision makers collaborate The Minister dominated the feedback. For the rest it was a question of with, discuss or provide feedback on internalizing the ICA findings rather than any questioning or pushback. the product? Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? The American University of Cairo was the contractor for the survey but did not contribute to the design of the second ICA. …analyze the results and write the No. report? …provide peer review or comments Although invited to the meetings AUC did not contribute much. on the draft report? …organize workshops or discussions The Ministry organized the discussion. about the findings? Discussion of Strategic relevance and ownership The interest and ownership of the Government remained strong for the second (2009) ICA. The work was consistent with both the Government priority and Bank assistance strategy. The rating under this criterion is Satisfactory. C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. make use of Fully used the existing ICA methodology. current and relevant knowledge available)? 72 Comments …cite relevant examples of practice or research There are the usual cross country comparisons though from other countries? these were of less interest to the authorities than the inter-temporal comparisons within Egypt. …discuss the specific institutional and policy A very important part of the discussion in the ICAs context for the issue? particularly in regard to regulation. …collect and analyze existing local data? …generate new evidence? Obviously the ICA represented a new data set. There are other enterprise censuses and surveys, but none geared to investor perceptions. The 2009 ICA also collected new data on upper Egypt and the informal sector and included coverage of services. …include recommendations? As indicated the recommendations were useful in agenda setting, but required more specific follow up ESW to derive actionable approaches. Is the policy advice consistent with the preservation This question deserves a book rather than a few of social order? sentences. The authorities clearly thought that the analysis and especially the treatment of corruption issues was potentially dangerous to the regime and should be closely held. On the other hand with hindsight had there been more openness about the presence of corruption and an associated willingness to take on the issue perhaps this might have contributed to a more orderly reform process. Do the recommendations include specific actions to Yes, but obviously these need follow up to provide be taken by specific actors? practical proposals for implementation. The Egyptians did not necessarily see this as the role of the Bank however. The diagnostics were what they were after. Given their own knowledge of the political economy and the delicate relations between ministries, they could then work out the best way to proceed on any given issue. Was the task team staffed with the necessary The task team drew on Bank sectoral analysis and expertise? expertise. Did the product receive adequate managerial The Country Director was fully engaged in the attention? discussion of both ICAs and saw this as a key ESW product given the Minister‘s interest. Did the product receive a sufficient budget? Yes Review of contents Was the draft peer-reviewed by appropriate experts? As with all the ICAs the peer review group includes peer reviewers drawn from a panel of ICA practitioners. Were the peer-review comments taken into account? Was the feedback from the client incorporated? Yes, particularly the views of the Minister. 73 Discussion of Technical quality Technical Quality is rated Satisfactory. A particularly interesting feature was the inclusion of questions on the availability of skilled labor. This was not generated initially from the firm interviews, but was a consequence of the Bank‘s sector work on education and vocational training. As a consequence the ICA looked at this issue and validated the concerns that Egypt‘s education system was not geared to the quantity and quality of vocational skills needed by the expanding business and services sectors. This is a very good example of effective synthesizing of analysis from different sectors – often a weakness in the Bank‘s approach. The ICA is very well presented, readable with well-chosen comparisons – unlike some where the comparisons overwhelm the reader. It was evident from discussions in Cairo that there was limited interest in the international comparisons – a contrast with the DBIs. The real concern here was to track the improvements in perceptions over time. Given the good technical quality it is even more disappointing that the reports have not reached a wider audience. D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local language? …made available on a website? Need to distinguish the report as a product, from the survey. The survey is technically available on the web, but not easily accessible. …discussed with senior policy-makers? Although the only direct discussion was with the Minister, he in turn used the ICA in discussions with other ministers. …presented at a workshop, conference or on The ICA was presented in a workshop and the team made the web? efforts to do other powerpoint presentations to reach a wider audience. …covered in the media? Some coverage but limited by the lack of publication. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? The Ministry of Investments became, along with the Ministry of Finance, the Bank‘s key counterparts and the ICA provided a basis for the ongoing dialogue. …discussions/contact with stakeholders? The TTL made a significant effort to reach out to other audiences …lending and non-lending products? There are a number of follow up analytical products and the ICA contributed to the DPL. Dissemination and sustained dialogue Dissemination and sustained dialogue are rated Moderately Unsatisfactory. For the 2009 ICA, there was a large dissemination event held, with the then MD of the Bank present. In addition there was dissemination outside Cairo. Once again this was all on the basis of Powerpoint presentations. Undoubtedly the fact that the TTL was based in Cairo contributed to the dissemination effort. But like the first ICA there was no public dissemination of the report. As far as the donor community is concerned the relevant donors are probably USAID and the African Development Bank. There was little interest. The OECD has recently carried out its own ICA with findings broadly consistent with the Bank‘s. 75 ANNEX 3: GUATEMALA Performance Assessment Report on Investment Climate Assessments Guatemala Investment Climate Assessment 2004 Report Vice President Pamela Cox Country Director Jane Armitage Sector Manager Susan Goldmark Task Manager Geeta Batra (in Report) and Heidi M. Mattila (in ACS) Project ID/ Type/Cost P083167 / ESW Climate Assessment/$64,123.61 Number of: Volumes/ Pages One volume/ 121 pages Languages English Document Type Investment Climate Assessment Topics Private Sector Development Unit LCSPSD Guatemala Investment Climate Assessment 2008 Report Vice President Pamela Cox Country Director Laura Frigenti Sector Director, Marcelo Giugale Sector Manager Lily L. Chu Task Manager Stefka Slavova Project ID/ Type/ Cost P104173 / ESW Climate Assessment/$134,930.14 Number of: Volumes/ Pages Two Volumes/ 151 pages Languages English Document Type Investment Climate Assessment Topics Private Sector Development Unit LCC2C-CMU Background After signing the Peace Accords in 1996, the government of Alvaro Arzú started reforms designed to accelerate growth and reduce inequality. Increasing productivity was one of the strategies pursued. The Bank sought to help by granting a loan of $20 million for a competitiveness project in the year 2000 – the final year of the Arzú administration. The loan entailed a survey of Guatemalan firms to ascertain their competitiveness. The FPD staff in the Region saw it as an opportunity to prepare an ICA. The administration that followed (Alfonso Portillo), however, did not implement the loan as planned, leaving the project and the proposed ICA in limbo. Then, in early 2003, the five nations of Central America (Costa Rica, El Salvador, Guatemala, Honduras, and Nicaragua) embarked on negotiations for a free trade agreement with the United States. The resulting CAFTA agreement was concluded and became effective in August 2004. The preparatory work for the CAFTA made the Government realize that it had to improve its investment climate to benefit from the agreement. The proposed ICA was now seen as an appropriate tool. The report would focus on information at the firm level in four areas: (i) regulatory environment; (ii) access to credit; (iii) quality of infrastructure; (iv) innovation, technology and human capital. The expectation was that the analysis would help identify essential reforms to promote growth through private and foreign investment. The report, completed one year later than planned, was published in December 2004 under the new administration of president Oscar Berger (January 14, 2004-January 14, 2008). The analysis in the report relied primarily on data from the urban manufacturing sector, but not the information available on 200 enterprises outside of the urban area. The report had an executive summary, five chapters, and three annexes, one of which summarized the information by size of firm and market orientation. The executive summary discussed the main 76 findings and econometric results on determinants of productivity in Guatemala, along with comparative results from Honduras and Nicaragua. The report proved valuable to the Government. The findings were used to raise the awareness of constraints on competitiveness and to establish a baseline for measuring progress. With the help of the report, the government prepared a competitiveness agenda for 2005-201, which had six strategic pillars. As the government pursued the reforms, it requested the Bank to do a follow-up ICA. The Bank carried out the enterprise survey and presented its preliminary findings of the second ICA to the government in July 2007. The final draft, however, was not available until February 2008, when president Alvaro Colom had replaced Oscar Berger as President. The final report was published in June 2008 – 14 months behind schedule. The 2008 ICA had an executive summary, five chapters, and recommendations. The executive summary discussed issues in the investment climate and recommendations of general nature. The report showed improvements in the investment climate between 2003 and 2007, but with retrogression in some areas. Overall, the report presented good information and analysis, but its impact on policy was modest. Overall Assessment With the 2004 ICA, the Bank produced a report of high strategic relevance, the results of which was used by the government to define an agenda for competitiveness. During the 2004-07 period, the government carried out reforms that led to a better investment climate, as documented by the 2008 ICA. The report was also of good technical quality, with the use of appropriate technical specialists both for the survey and the analysis. Statistical information from the survey was analyzed along with international data to determine the competitive position of Guatemala. But its recommendations were too general to lead to specific reforms. The Government, however, was able to overcome the problem by treating the generic recommendations as an identification of priorities and by developing its own specific actions. Furthermore, the Bank did not provide Spanish translation and thus limited its ability to reach a wide audience. In brief, the report earned good marks for its strategic relevance, as well as results, but was not so strong on technical quality and dissemination. The 2008 ICA showed similar strengths and weaknesses, but the results achieved were more modest by far. The difference reflects a change in the attitude and priority of the incumbent Government. The Colom administration which started as the report was being finalized placed greater value on social cohesion than on competitiveness. As a result, the reforms initiated by the previous administration fell out of favor. Respecting the priority of the new administration, the Bank did not actively pursue the old agenda or widely disseminate the report. The country dialogue became more subdued and less active, as was the Bank‘s assistance program in the country. Lessons The experience of these two ICAs gives rise to the following lessons. First, chance, luck and timing play an important role in shaping the outcome. Despite the Portillo administration‘s indifference, the investment climate, and with it the Bank‘s program of assistance, took on a new meaning with the onset of the negotiations for CAFTA. The interest in the topic surged during the Berger administration. The timing of the 2004 ICA was perfect. The Bank‘s readiness to respond created an opportunity to make a lasting impact. The 2008 ICA met with the opposite fate. It started with great enthusiasm at the request of president Berger, but ended quietly with little or no results, largely because it was delivered to an administration (of President Colom) that gave the investment climate a low priority. Second, the 2004 ICA shows, strong government ownership can overcome serious shortcomings in Bank advice. Despite the lack of indigenous knowledge and specificity of actions in the report, the Berger administration was able to find a solution and work out a detailed competitiveness agenda. Third, the Bank‘s dialogue and dissemination of knowledge products should proceed on a regular basis, without long interruptions. (The products involved, of course, should be – to the extent possible – ideologically neutral.) The Bank‘s retreat from an active engagement during the Colom administration is seen by in-country partners as a costly mistake – one that carries a high cost in terms of grassroots support and public opinions in favor of reforms. 77 Forth, there is scope for greater sensitivity to the needs of the local audience, especially when it is engaged in knowledge work. The two ICAs which were available only as full-length reports in English reached a limited number of readers and missed the opportunity to influence public debate. Meanwhile, the Inter-American Development Bank was able to provide the audience with similar reports in Spanish and in abridged versions. Ratings Summary ICA 2004 Criterion Rating Comment Results Satisfactory The ICA raised the awareness of issues in the investment climate, generated useful information including a baseline for future progress and contributed to the preparation of the Government‘s agenda for competitiveness. It was also used as input for the FY05 CAS on Bank support for growth and competitiveness. Strategic Highly The Berger administration saw great value in the report and used it relevance Satisfactory extensively in policy formulation. The ICA was an integral part of the Bank‘s assistance strategy, had a well-defined rationale and the client was engaged with it. Technical Satisfactory The technical quality was good, although the recommendations were not quality immediately actionable. Dialogue and Satisfactory The findings of the report were well disseminated, but could have reached a dissemination wider audience if it had been translated into Spanish and adapted in length for different target groups. ICA 2008 Criterion Rating Comment Results Moderately The report had a modest impact on the ground in terms of reforms, but the Unsatisfactory findings informed Bank operations. Strategic Moderately The new Government showed little interest in the report and the investment relevance Satisfactory climate in general, although it was consistent with the Bank‘s assistance strategy. Technical Satisfactory The technical quality was good, but the recommendations showed limited quality appreciation of the political economy. Dialogue and Moderately The Bank discussed the report with the main counterparts, but did not dissemination Unsatisfactory disseminate the report to other stakeholders or the general public. 78 Detailed Assessments: Guatemala 2004 ICA A. Results Comments Results indicators Did the product have: …clear results / objectives defined at Not explicitly, but it can be derived from the executive summary and inception? from Box 1.1 that explains what an ICA is. In the executive summary, the report asks ―which features of Guatemala‘s investment climate pose particular obstacles to economic growth and development?‖ To answer that question, the ICA used data from surveys of 455 manufacturing enterprises, 250 enterprises in the rural and informal sector; and other publicly available sources with ―The hope is that the results will help identify the reforms most critical to private sector development and facilitate consensus on a more far- reaching agenda of reform.‖ (par. i). Box 1.1 states that ―By providing a practical foundation for policy recommendations and involving local partners throughout the process, the assessments are designed to give greater impetus to policy reforms that can speed the private sector‘s growth, leading to faster economic growth and poverty reduction.‖, and ―The findings of the survey, combined with relevant information from other sources, provide a practical basis for identifying the most important areas for reform aimed at improving the investment climate.‖ …clearly defined indicators defined at Yes. They were defined in the survey questionnaires which sought to inception? generate information on (i) the regulatory environment (policy and regulatory uncertainty, red tape, corruption and crime), (ii) access to and costs of credit, (iii) cost and quality of infrastructure services and logistics costs; and (iv) low levels of investment and activity in innovation, technology absorption and human capital formation. …a clear strategy to achieve results The report presented a table listing deficiencies in the investment (log-frame)? climate and recommending actions to deal with them, but many actions had a general tone that made it difficult to know the specific steps that would tackle the deficiency. Among other studies, the ICA report helped PRONACOM (national competitiveness program) prepared and monitor a national competitiveness agenda. One person interviewed noted that the ICA had clear indicators but lacked a practical strategy to reach them, and did not know whether this obeyed to a failure of design or of implementation. …results that the client understood? Yes. The findings supported the Berger government‘s agenda for competitiveness (―agenda de competitividad‖) that incorporated some of the ICA‘s findings. The Berger government was in power during 2004-2007 Results achieved In the client country, were the findings used… …in government policy law or The government promoted a law on public/private partnerships to regulation? promote foreign investment, and it used the findings to carry out reforms in the regulatory environment and in infrastructure, telecoms especially, which led to a better investment climate in 2004-2007. The ICA 2008 documents the improvements. 79 …in design of spending or investment No programs? …to raise stakeholder awareness? Different media disseminated the assessment‘s findings, and the public, which knew of the problems, became aware of their magnitude. The study helped raise awareness of the importance of regulation, its costs and benefits. …to build a coalition for change? Yes. The government appointed a competitiveness commissioner (Comisionado de Competitividad) who, in collaboration with the private sector and other members of civil society, drew an agenda for competitiveness, identified areas for change (e.g., policies, infrastructure), and monitored the changes carried out. Out of this grew a group that met about once a month to follow up on commitments and to identify implementation problems and ways to deal with them. …to influence the donor community? Not directly, but indirectly through the discussions of the CAS. In the Bank, were the findings of the product used in …Bank strategy formulation The CAS (FY05) used the ICA‘s findings, together with a 2004 (CAS/PRSP)? CEM, for its pillar Guate Crece/Compite (Guatemala grows and competes) that focused on investment climate issues hindering growth. The CAS progress report (FY07) highlighted the investment climate as one government priority and planned a second ICA for FY08. …the design of Bank lending products One loan (FY01) supported the national competitiveness agenda, and or AAAs? the CAS envisaged granting three development policy loans (DPLs), a tax administration TAL, infrastructure guarantees, and doing a CAFTA regional integration study; the loans would support areas that the ICA covered. The three DPLs covered, among others, the financial and general industry and trade sectors, with the purpose of influencing regulation and competition policy, tax policy and administration, trade facilitation and market access, and infrastructure services for private sector development, all of these ICA themes. If there were recorded results, is there Yes, for some. The reductions in barriers to doing business and the evidence that they are likely to be changes in the telecommunications sector are likely to stay, but the sustainable? improvements in infrastructure and the advances in crime have suffered a setback as a result of the change in priorities towards social sectors in the Colom government (2008-2011) and the inroads of drug trafficking. . Monitoring of impact Did the Bank or the client undertake an The Bank carried out a second ICA at the request of the government, assessment of the product‘s impact? and the Berger government appointed a competitiveness commissioner to prepare a competitiveness agenda and monitor it. Neither the Bank nor the client seem to have taken steps to find out the impact of the changes carried out. What intermediate outcomes or As the ICA 2008 notes, perceptions on the business environment indicators reflect the influences of the improved. Firms perceived improvements in corruption, crime, ICA? taxation and tax administration, macroeconomic stability, customs and trade regulations, and the functioning of courts. 80 What results / outcomes are attributable It is difficult to trace a particular result to the ICA, because the to this ICA? authorities used other sources like the Doing Business and World Competitiveness reports to find out how they were doing and where policy could be improved. From the interviews it came out clearly that the Berger administration valued this tool, when it requested a follow-up ICA to find out how much the investment climate had improved. Discussion of Results The ICA 2004 served to raise awareness of the extent and magnitude of the problems firms faced and to guide the Bank‘s country strategy of 2005 in one of its pillars, Guate Crece and Compite. It also helped the government to formulate its competitiveness agenda (Agenda de Competitividad) and set baseline values to monitor progress in its plans to improve the investment climate. For some the ICA‘s value added rests in its comparative nature with other countries and in defining a benchmark, but one major shortcoming, with which this review agrees, are its generic recommendations and on the policies to follow and the studies that should be done as a follow up. B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. Comments Strategic relevance Was the delivery of the output timely and on No. The Bank delivered the product in August 2004, 11 months schedule? later than expected. The final report was issued in December 2004. Was the issue identified as a development constraint in (note all that apply) …the relevant CAS? Yes. The 2005 CAS had one pillar devoted to growth and competitiveness and the factors that influenced them. …previous lending or non-lending work? In its lending and non-lending work the Bank dealt with several of the topics the ICA touched. In lending, the Bank granted loans on land administration (FY09), competitiveness (FY01), basic education (FY01), financial sector (FY02), and rural infrastructure (FY03). In non-lending the CEM (FY05) and the poverty assessment (FY03) are the most notable pieces, but other Bank publications and working papers touched on the issues that the ICA covered. Among these are a FSAP in FY02, not disclosed to the public, the role and impact of the Central America trade agreement with the US (FY03, infrastructure and its impact on poverty (FY04), payments systems (FY04), judicial modernization (FY05), education and economic growth (FY05), barriers to quality in education (FY05), financial exclusion (2005) and a diagnosis on transparency and corruption (2005). …policy dialogue with clients? The lending and non-lending work reflects the widespread range of topics in the dialogue with the authorities, and which the signing of the Peace Agreements in 1996 had strengthened. …donor coordination forums? Some of these topics were discussed in donor coordination forums, as they had to do with economic growth, but after the Peace Accords donor assistance was conditioned to carrying out a land census, reforming the tax structure and the judiciary, and starting programs to title land and reduce rural poverty. 81 Comments In cases where the dialogue is difficult, did Both the government and the Bank thought that solving the product, did the task find a solution? problems of competitiveness and productivity were essential for Guatemala‘s development. The competitiveness project, which had been approved during the Arzú government (1996-2000), suffered in its implementation during the Portillo government, and had to be reformed in 2005; the findings of the ICA helped in the loan‘s reform. Ownership Is the product part of an overall program of The ICA grew out of a work on labor markets that had started in AAA to which the authorities have the late 1990s and of a survey on competitiveness that was to be contributed or agreed? done for the competitiveness loan. Both government staff under the Arzú administration (1996-2000) and the Bank wanted the survey. After Alfonso Portillo came to power (2000-04) the loan‘s implementation stalled. The Oscar Berger administration (2004-08) asked for a restructuring of the loan, and only then its implementation started. Did the client request or commission the Yes, because it would support the Bank‘s competitiveness loan, specific product? which the authorities were interested in. Did the client cover some or all of the costs? No. Did key decision makers collaborate with, Yes. discuss or provide feedback on the product? Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? A local consulting firm (GSI) did the survey for the Bank and CIEN, a think tank, did the supply side of the study for the labor market, …analyze the results and write the report? See comment above …provide peer review or comments on the The Bank discussed the report with the authorities and in draft report? particular with people that would be involved in the Berger administration (2000-2004) in different capacities (e.g., advisors, competitive commissioners, FUNDESA –a private think tank-, and staff of PRONACOM and Invest in Guatemala). …organize workshops or discussions about The competitiveness commissioner organized meetings with the findings? government officials, private sector representatives, and other members of civil society to define and monitor the government‘s competitiveness agenda, which was launched in September 2005. Regional and sectoral workshops helped shape the agenda, and the information in ICA helped to ground the discussion in the workshops. A broad council monitored the ICA, and the Bank participated early on in one of them to present the main conclusions of the ICA. The agenda was important for the Berger government, to the point that staff in the competitive commissioner‘s office participated in the economic cabinet. 82 Discussion of Strategic relevance and ownership From the Guatemalan viewpoint the study achieved its main goal of raising awareness. It did it well by identifying priorities and critical themes. Before the ICA was done, Guatemalans used the World Economic Forum (WEF) indicators as their reference point. After its completion the ICA generated much interest. Just the process of gathering the information made the ICA more relevant than the WEF in the view of some of those interviewed. The ICA produced more detailed information than the WEF and Guatemalans saw its indicators and themes as more relevant than those of the WEF. Moreover, the ICA managed to get the collaboration of relevant actors in Guatemala. In summary, the ICA was an integral part of the Bank assistance strategy, had a well-defined rationale and the client was engaged with it. C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. The report discusses what affects productivity in the 455 firms make use of current and relevant surveyed (executive summary), explains why investment climate knowledge available)? matters (Chapter 1), and then presents its findings on regulation, infrastructure, innovation, quality and skills (Chapters 2-5). The discussion on productivity summarizes econometric results (Tables 1 and 2 of the report) based on an extensive paper by Escribano and Guasch (wps 3621, 2005) that explores the determinants of productivity in Guatemala, Honduras and Nicaragua using up-to- date techniques. Chapter 1could have explained how a good investment climate promotes real costs reductions, along the lines of A. C. Harberger‘s presidential address to the AEA (AER, March 1998), thereby linking better the findings of Chapters 2-5 to growth accounting and economic theory and, also, giving better reasons for concentrating on regulation, infrastructure, finance, and innovation and skills. A former government staff and now private investor questioned some country comparisons on the ground that the data are not comparable because of the way the statistical agencies gather their information. He also pointed out that as an investor the rule of law is fundamental, and when he contemplates investing in a country he spends much time talking to lawyers about the legal pros and cons of going there; other aspects he looks at are infrastructure and education, two variables that the ICA looks at. He also suggested the Bank should talk to investors to learn more about which variables to include in the surveys, as some might be irrelevant, as his experience in government taught him. …cite relevant examples of practice or The assessment compared the situation of Guatemala with other research from other countries? countries in four dimensions, and presented a detailed analysis of the determinants of factor productivity in Guatemala, Nicaragua and Honduras based on the same surveys in the three countries. The report also discussed the low skills equilibrium model describing the UK in the early 1990s and its application to Guatemala, and summarized Chile‘s experience in changing its organizations and institutions to provide better vocational education and training. …discuss the specific institutional and The report touched upon institutional issues indirectly when it policy context for the issue? informed about perceptions (e.g., corruption, government effectiveness) or measured the extent of a problem (e.g., children 83 Comments enrolled in primary school, time spent dealing with regulators) but not on the potential causes of the problems (e.g., excessive regulation or insufficient budget for education). Not covering institutions and policies may explain why the recommendations for change tend to be general rather than specific. On this aspect, some people interviewed noted that the government follows more closely the Doing Business report because it tends to be more specific and is published annually. …collect and analyze existing local A local firm (GSI) did the survey of 455 firms and another one, data? CID Gallup, surveyed 250 enterprises in the rural and informal sector. The report also benefited from information of Guatemala‘s statistical agency (INE), but most of the information presented comes from surveys carried out in Guatemala and other comparator countries. …generate new evidence? The report informed about the extent and magnitude of the problems affecting the investment climate in the four dimensions it set to analyze: regulatory environment, access to and cost of credit, cost and quality of infrastructure, and investment in innovation, technology absorption and human capital formation. The information was not available before, and became a baseline on which the government and the public could judge whether reform had led to better institutions and a more favorable investment climate. …include recommendations? The report recommended measures for dealing with the deficiencies in the investment climate (Table I.1) but, as noted in the comments to the third question of this section, they were general (e.g., continue efforts to modernize the judicial system) with little guidance on the specific steps to follow. ….give policy advice that is dovetailed Because of their general nature, the recommendations did not (and to the local political economy? did not have to) dovetail to the local political economy. The government, though, converted these general recommendations into specific actions when it prepared its competitiveness agenda. Do the recommendations include No. The report recommends general actions (e.g., greater specific actions to be taken by specific transparency, improve education) and does not identify specific actors? actors to carry them out. Was the product team staffed with the The people interviewed in Guatemala noted the good technical appropriate expertise? quality of the team that prepared the report and its different components. Did the product receive appropriate managerial attention? Did the product receive a sufficient Information on this is scanty. The activity completion summary budget? (ACS) puts the cost of the survey at $49,239.15 and a Business Warehouse report lists the cost of the product at $15,000. These sums seem to underestimate the cost of producing this type of report, if one were to judge it by the number of people that participated in its preparation and by the one year that it took to deliver the survey to the client. 84 Comments Review of contents Was the draft peer-reviewed by The people that reviewed the report knew well the region, the appropriate experts? country and the methodology and conceptual underpinnings of the ICAs. Were the peer-review comments taken into account? ` Interviewees in Guatemala did not mention it, but the report was discussed with officials of a government that wanted to improve the investment climate and with stakeholders in the private sector with the same goal. FUNDESA, a private think-tank, and PRONACOM, the organization in charge of the competitiveness agenda, commented on the report and the Bank accepted them with an open mind. Discussion of Technical Quality Guatemalans gave good marks to the ICA, both for its credibility and quality (mostly 5 in a scale of 1-6 where 6 is best). These characteristics contributed to making the ICA‘s findings a benchmark to monitor the competitiveness agenda. The report served to show where Guatemala was in terms of investment climate and how it compared to similar countries. This reviewer notes that the numbers quoted in the executive summary overestimate impact of each dimension (e.g., red tape or infrastructure) because the author adds the estimated coefficients shown in Tables 1 and 2 no matter whether they are significant from a statistical viewpoint. While Guatemalans think well of the report‘s technical quality, they noted that (a) the general nature of the recommendations did not guide them on what to do in practice, and (b) policymakers did not have the time to read a report as long as the ICA. The general recommendations point out at the lack of knowledge of organization and institutions in Guatemala, and the weak grasp of the political economy of the country. The latter was not a serious constraint for this report because those monitoring the competitiveness agenda organized monthly meetings with 60-70 members of the private sector to track progress in the six axes of the agenda, to identify bottlenecks to change, and to propose solutions for them. No one commented about the quality of writing and editing, but the frequent comments on the report‘s length indicate that writing and editing could have been better. D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local No. This is a significant shortcoming in the Bank‘s efforts to disseminate language? the product, as few people can speak or read English. Moreover, some interviewees noted that the Bank should produce several versions of the report (a complete one with the full technical analysis, a shorter version - -20 pages- with the main points and some technical information, and a short one -4 pages- for busy, high-level people who have only the time to read short notes). …made available on a website? In the Bank‘s external website, but in Guatemala having it in the web does not mean much. One person interviewed noted that people in Guatemala want a hard copy and a CD of the product; as good example of dissemination he gave the UNDP‘s Human Development Report for Guatemala. The UNDP office in Guatemala invites all stakeholders, 85 including the President, to a meeting where with great fanfare it distributes the book and the CD to all participants. It then goes to other cities in Guatemala to present the report. He pointed out that most of the people going to these presentations do not bother to look at the website where the report could be downloaded. …discussed with senior policy- The Portillo government paid little attention to investment climate issues, makers? but the Berger government asked for Bank help besides lending. The ICA was one of the works that fitted well in the government‘s agenda, and the report was discussed with the government in 2005 and its main conclusions were presented to the Consejo Ampliado that monitored the competitiveness agenda. …presented at a workshop, In addition to a presentation to the Consejo Ampliado de Competitividad, conference or on the web? no other dissemination took place. …covered in the media? One interviewee involved with the product since the beginning noted that the media covered the topic widely, making people aware of the type and magnitude of the problems affecting Guatemala‘s investment climate. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? This happened while the government showed interest in the topics the ICA covered …discussions/contact with After the initial presentations to Guatemalans the Bank did not follow up stakeholders? on the fate of its recommendations, but PRONACOM and the Competitiveness Commissioner met regularly with 60-70 stakeholders to monitor progress in that agenda. …lending and non-lending Some loans that followed the ICA supported the recommendations of the products? ICA through policy conditions in the DPLs or specific reforms in investment loans. …other means (please discuss below)? Discussion of Dissemination and Sustained Dialogue Guatemalans gave the Bank high scores for the strategic relevance and the technical quality of the report, but all agreed that its dissemination strategy could have been better. First, several pointed out the need to have the report in Spanish, with some suggesting the Bank should write it first in Spanish and then translate it into English. Second, the reports are too technical and the Bank needs to package it in different shapes to reach different audiences, as they have different needs, interests, and time to read it. One suggested producing three versions: the full report, a mid-size version (20 pages) for people with some time to read it and interest in technical aspects, and a short version (4 pages) for policymakers who have little time to read and neither can nor need to absorb most of the technical aspects of the report. Despite these shortcomings the media covered the report‘s findings and the report reached PRONACOM and the Competitiveness Commissioner, the ones in charge of designing, executing, and monitoring the competitiveness agenda. The Bank had a chance to present and discuss the findings and conclusions of the report with the Consejo Ampliado de Competitividad (Broad Council on Competitiveness), a large group of public and private sector people who diagnosed problems, agreed on solutions and monitored advances in the agenda; they used the report‘s findings as baseline values and benchmark against other countries. 86 Detailed Assessment: Guatemala 2008 ICA A. Results Comments Results indicators Did the product have: …clear results / objectives defined at Not explicitly. The 2008 ICA report came out of a request inception? from the Berger government to learn how governance and regulation, skills and technology, access to infrastructure services, access to financial services, and firm productivity had changed after the government started implementing its competitiveness agenda. The report continued in the footsteps of the 2004 ICA report, and sought to document the extent of change in the investment climate and to identify the most serious constraints to a good investment climate. …clearly defined indicators defined at Yes. They were defined in the survey questionnaires which inception? sought to generate information on (i) governance (red tape, corruption, land rights, courts and contract enforcement, crime and insecurity); (ii) access to financial services; (iii) access to infrastructure services; and (iv) constraints to exports. …a clear strategy to achieve results (log- The report presented its recommendations in a summary frame)? table at the end of Volume 1. The recommendations were general and there was not one single recommendation that could be associated to the solution of a specific problem, in part because the problems were also defined in a general way (e.g., public expenditure in roads is inadequate and roads are vulnerable to natural disasters). …results that the client understood? The report recorded the change in the investment climate in Guatemala between 2003 and 2007 (the dates of the enterprise surveys), showing where it had improved and where it had deteriorated. It did it in a clear manner and anyone interested in its results and able to read English could follow it with ease. Results achieved In the client country, were the findings used… …in government policy law or regulation? No. The business climate as a policy issue lost importance during the present government (Alvaro Colom, 2008-2011). …in design of spending or investment No programs? …to raise stakeholder awareness? No …to build a coalition for change? No …to influence the donor community? No 87 In the Bank, were the findings of the product used in …Bank strategy formulation (CAS/PRSP)? There has not been a CAS or PRSP after the ICA 2008 was completed. …the design of Bank lending products or One loan under negotiation, financing for SMEs, used the AAAs? results of the report as input for its preparation. If there were recorded results, is there evidence There are no results that can be associated with this ICA. that they are likely to be sustainable? One of the interviewees mentioned that the ICA 2008 report did not go beyond the office of the Ministry of Finance, and that the head of PRONACOM had neither seen the report nor used it as input for the design of the new competitiveness agenda under preparation. Monitoring of impact Did the Bank or the client undertake an No assessment of the product‘s impact? What intermediate outcomes or indicators None reflect the influences of the ICA? What results / outcomes are attributable to this None ICA? Discussion of Results There are no results to discuss B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. Comments Strategic relevance Was the delivery of the output timely and No. The Bank delivered the product in June 2008, 14 months later on schedule? than the planned original date. Was the issue identified as a development constraint in (note all that apply) …the relevant CAS? Yes. The 2005 CAS had one pillar devoted to growth and competitiveness and the factors that influenced them and the CAS Progress Report of 2007 mentions the investment climate as one element of that pillar. …previous lending or non-lending In its lending and non-lending work the Bank dealt with several of work? the topics the ICA touched upon. In lending, the Bank granted loans on land administration (FY09), competitiveness (FY01), basic education (FY01), financial sector (FY02), and rural infrastructure (FY03). In non-lending the CEM (FY05), the poverty assessment (FY03) discussed the investment climate, and other Bank publications and working papers touched on the issues that the ICA 88 Comments covered. Among the publications and working papers are a FSAP in FY02, not disclosed to the public, an update of the FSAP (FY07), the role and impact of the Central America trade agreement with the US (FY03, infrastructure and its impact on poverty (FY04), payments systems (FY04), judicial modernization (FY05), education and economic growth (FY05), barriers to quality in education (FY05), financial exclusion (2005) and a diagnosis on transparency and corruption (2005). All this work was carried out before the Colom administration which has shown little interest in the topic. …policy dialogue with clients? No …donor coordination forums? No In cases where the dialogue is difficult, No. Dialogue on this topic has been minimal. did the product, did the task find a solution? Ownership Is the product part of an overall program Yes. It was contemplated in the CAS Progress Report of 2007. of AAA to which the authorities have contributed or agreed? Did the client request or commission the Yes. The Berger government requested a follow-up report to the specific product? 2004 ICA. Did the client cover some or all of the No. costs? Did key decision makers collaborate The Bank presented the final report to the Ministry of Finance in with, discuss or provide feedback on the February 2008, but the report did not go beyond that office. product? Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? A local consulting firm (GSI) did the enterprise survey. …analyze the results and write the No report? …provide peer review or comments on In July 2007 the Bank presented a preliminary version of the report the draft report? to the Ministries of Finance and Economy, FUNDESA -a private think tank-, the National Competitiveness Program (PRONACOM), and the National Competitiveness Council. People in these organizations provided feedback and the report incorporated their comments. …organize workshops or discussions The Ministry of Finance held a meeting to discuss the final report, about the findings? but there were no other meetings to discuss it. Discussion of Strategic relevance and ownership The ICA 2008 had negligible relevance because the Alvaro Colom government set its sights on strengthening the social agenda (e.g., conditional cash transfers) and gave low priority to investment climate issues. Even if delivered as scheduled the report would have had little impact because 2007 was an electoral year and the scope for reform was limited. 89 C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. The report summarizes the findings of the survey (Chapter 1), make use of current and relevant discusses the impact of CAFTA on firms, and reviews the situation of knowledge available)? firms‘ access to infrastructure and financial services (Chapters 3 and 5), and surveys the governance situation (red tape and corruption, land, courts and contract enforcement, and crime and insecurity). It also has a summary table on recommendations. The Bank has used the information gathered for this report to prepare a report on the situation of the small and medium enterprises in Guatemala which PRONACOM has been using as an input for its competitiveness agenda for the next ten years. …cite relevant examples of practice The report brings a substantial amount of information about or research from other countries? investment climate in other Latin American countries, placing in perspective how Guatemala compares to other countries in the region. The report includes information on Chile, a country with institutions substantially different from the Guatemalan ones; Chile was included in the survey at the request of the government who wanted to know how far it was from a good performer. …discuss the specific institutional The report touched upon institutional issues indirectly when it and policy context for the issue? informed about perceptions (e.g., red tape, contract enforcement) and measured the extent of a problem or firms perceptions about it, but it did not specific aspects of policies, society‘s attitudes, and organizations leading to the outcomes recorded in the survey. Neglecting this side of the investment climate explains why the recommendations for change tend to be general. …collect and analyze existing local A local firm (GSI) did the survey. The report also benefited from data? information of Guatemala‘s statistical agency (INE). …generate new evidence? Yes. The report recorded the situation of the investment climate in 2007 and measured the changes in the investment climate during 2003-2007. Most of the changes indicated an improvement in that climate. …include recommendations? The report recommended general actions (e.g., improve accounting and auditing practices) to improve the investment climate (pages 84- 88), and offered no guidance on specific steps. ….give policy advice that is As in the ICA 2004 the report‘s policy advice is not dovetailed to the dovetailed to the local political local political economy because its recommendations are too general; economy? their general nature makes the report less attractive to a government that shows little interest for investment climate issues. Do the recommendations include The report recommends general actions (e.g., design and implement specific actions to be taken by specific strategy to improve security for domestic and international shipments, actors? improve efficiency of targeted subsidies for electricity, and others along similar lines). Was the product team staffed with the The people interviewed in Guatemala noted the good technical quality appropriate expertise? of the team that prepared the report and its different components. 90 Did the product receive appropriate managerial attention? Did the product receive a sufficient The activity completion summary (ACS) puts the cost of the report at budget? $135,000; this information is consistent with that in the Business Warehouse. Review of contents Was the draft peer-reviewed by The peer-reviewers knew well the region, the Bank‘s investment appropriate experts? climate program and the problems affecting private sector development. Were the peer-review comments taken into account? Was the feedback from the client A first draft of the report was discussed with officials of the Berger incorporated? administration in the Ministries of Finance and Economy, in PRONACOM, the National Competitiveness Council, FUNDESA –a private think tank-, and with two of the Commissioners for Competitiveness who gave ideas for the report and supported the effort behind it. Discussion of Technical Quality Guatemalans gave good marks to the ICA, both for its credibility and quality (mostly 5 in a scale of 1-6 where 6 is best). The report did not become a benchmark for the competitiveness agenda because the Colom government (a) gave low priority to the agenda and (b) preferred to work with IDB, which had presented a comprehensive report of the problems affecting Guatemala. As for the ICA 2004 Guatemalans noted that while the report was of good technical quality, its length, the language, and the general nature of its recommendations made it less useful than it could have been otherwise. As for the ICA 2004, the general recommendations point out at the lack of knowledge of organization and institutions in Guatemala, and the weak grasp of the political economy of the country. Together with the low interest for investment climate issues, these problems possibly added to the report‘s low profile and contributed to its negligible influence. No one commented about the quality of writing and editing, but the frequent comments on the report‘s length and the convenience of having different types of report for different audiences indicate that the writing and editing could have been better. D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local No. This is a significant shortcoming in the Bank‘s efforts to disseminate language? the product, as few people can speak or read English. Several interviewees suggested that the Bank should produce versions of the report of different size to reach different audiences. …made available on a website? In the Bank‘s external website, but having it in the web does not mean much in Guatemala. One person interviewed noted that people in Guatemala want a hard copy and a CD of the product. As good example of dissemination he gave the UNDP‘s Human Development Report for Guatemala. The UNDP office in Guatemala invites all stakeholders, including the President, to a meeting where it distributes the book and the CD to all participants. It then goes to other cities in Guatemala to present 91 the report. He pointed out that most of the people going to these presentations do not bother to look at the website where the report could be downloaded. Another interviewee suggested that one reason for IDB‘s success with the Colom government came from having summarized its existing work on Guatemala and packaging in a report that was intelligible, easy to read and in Spanish. …discussed with senior policy- The Bank discussed this report with the Ministry of Finance in February makers? 2008. The report did not go beyond this office and the Bank did not make an effort to distribute the product or disseminate its findings …presented at a workshop, conference or on the web? …covered in the media? One interviewee involved with the product since the beginning noted that the media covered the topic widely, making people aware of the type and magnitude of the problems affecting Guatemala‘s investment climate. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? No. The government did not show interest in it and the Bank did not push it. …discussions/contact with No. The discussion of the report ended after the presentation to the stakeholders? Ministry of Finance in February 2008. One interviewee, commissioner of PRONACOM from mid-2008 to mid-2010 said to not know the report. Instead, he mentioned to have used the findings of the report on SME Development in Guatemala, completed in June 2008. He spoke highly of the quality of its analysis and information and did not understand why the Bank had not distributed it widely. …lending and non-lending One loan under negotiation, financing for SMEs, is based on the findings products? of the report. The Ministry of Economy, which has two loans under preparation with the Bank, has not used the report to define their content and components. …other means (please discuss No below)? Discussion of Dissemination and Sustained Dialogue Guatemalans who knew the report or knew of it gave the Bank high marks for its technical quality, but all agreed that no effort had been done at disseminating it. Some of the comments on dissemination about the ICA 2004 apply to the ICA 2008. First, the report should be in Spanish, some suggesting the Bank should write it first in Spanish and then translate it into English. Second, the Bank needs to package the report in different shapes and sizes to reach different audiences. One suggested producing three versions: the full report, a mid- size version (20 pages) for people with some time to read it and interest in technical aspects, and a short version (4 pages) for policymakers who lack that time. Guatemalans consider that the Bank has to learn how to package its products, and compared the experience of ICA 2008 with a report that IDB presented to the Colom administration in early 2008. IDB summarized in a report for the government several reports that it had done recently on the Guatemalan economy touching, among other, on investment climate issues. Third, Guatemalans felt that the Bank did not push the competitiveness agenda sufficiently hard, and believed that a more active country office could have counteracted some of the lack of interest of the government in that agenda. One person interviewed noted, though, that the report help to prevent a further deterioration of the agenda‘s status and the status of the organizations in charge of the competitiveness agenda, PRONACOM and Consejo Nacional de Competitividad. 93 ANNEX 4: KENYA Performance Assessment Report on Investment Climate Kenya Investment Climate Assessment 2004 Report Vice President Callisto E. Madavo Country Director Makhtar Diop Sector Manager Demba Ba Task Manager Vijaya Ramachandran Project ID/ Type/ Cost P080307/ESW/$135,000 Number of: Volumes/ Pages 1/90 Languages English Document Type ICA Topics Private Sector Development Unit Africa Finance and Private Sector Development Group - AFTPS Kenya Investment Climate Assessment 2008 Report Vice President Obiageli Katryn Ezekwesili Country Director Colin Bruce Sector Manager Marylou Uy Task Manager Giuseppe Iarossi Project ID/ Type/Cost P102213/ESW/ $419,000 Number of: Volumes/ Pages 1/121 Languages English Document Type ICA Topics Private Sector Development Unit Africa Finance & Private Sect Development -AFTFP Background Two ICAs were completed in Kenya, the first in 2004 and the second in 2008. In addition, several enterprise surveys had been conducted in addition to those used in these two reports. In part, the work in this area was stimulated by the Government‘s initial efforts to promote private businesses. Government interest in the private sector grew following the inauguration of the new National Rainbow Coalition Government in December 2002. The change was based on the recognition that the private sector would be the driver of economic growth and job creation. Deferred maintenance and years of neglect had left Kenya‘s infrastructure in disrepair. Railway services had collapsed, with its share in transportation falling from 60 percent in 1990 to 6 percent today. Airport capacity was unchanged while the number of passengers rose from 2.1 million to 5.1 million. Corruption was widespread, often preventing much needed projects from being implemented. Despite the new attention given to the private sector, including the implementation of the Economic Recovery Program, progress made on the investment climate was limited. The reforms envisaged in the private sector development strategy for 2006-2010 were disrupted by the civic strife following the contested elections of December 2007. The spread of violence depressed the economy, reducing growth in 2008 to less than 2 percent, from more than 7 percent in the previous year. The economic recovery program was re-launched in 2008 by the new coalition government, but the private sector component, which had the support of the EU and the UK DFID, remained unfinished. A new constitution was promulgated in 2010 with stronger anti-corruption framework, but it is too early to see the results. 94 Overall Assessment The 2004 ICA is relatively strong on technical quality, which reflects the Bank‘s expertise in conducting enterprise surveys. The report covers: a) An overview of macro trends and prospects; b) An analysis of firms‘ assets, investment, productivity growth, and manufacturing exports; c) A review of the competitiveness (i.e. productivity vs. costs) of labor; d) Key constraints to competitiveness, such as corruption, crime, infrastructure, as well as the risk of political instability; and e) Policies to address the constraints. Recommendations were mixed, however, including both actionable advice and generic remarks. The relevance of this ICA is based primarily on the alignment with the Bank‘s CAS and Kenya‘s objectives, but it also benefited from an early engagement with local partners including an institute that conducted the enterprise survey. The Government‘s ownership of the report, however, turned out to be tenuous, with discussions confined to the Ministry of Industry. In addition, the Bank‘s dialogue on this ICA was significantly constrained by the use of a large number of short-term consultants, each with narrow terms of reference. Not surprisingly, the report did not enjoy sustained interest of the Government and few of the ICA findings or recommendations were adopted. On the other hand, the report proved useful within the Bank, serving as inputs for both economic and sector work, as well as project preparation. The 2008 ICA has had substantial impact. The dialogue benefited from targeting the right audience, broad media coverage, and the use of an edited version that was shorter and more readable than the full-length output. The dialogue however focused primarily on dissemination, with little allowances made for local capacity building and follow-up support which could have enhanced the results. The strategic relevance is derived from its consistency with the Bank‘s 2008 CAS and its focus on key development issues. The Government also showed a greater interest than was the case with the previous report. The technical quality is also good, with a strong team conducting the enterprise survey and doing the underlying analysis for the report. The survey results were used in combination with cross country data to determine the competitive position of the country overtime and in relation to appropriate competitors both in the region and at the global level. Despite the relevance of the report, the technical quality and good dialogue, the results were significantly undermined by tragic events that engulfed the country in ethnic strife and protracted violence in 2008. Within the Bank, however, the report proved useful for both country dialogue and project work – particularly in transport and energy. Lessons One, the lack of social and political stability weakens results. While Kenya‘s government showed an interest in the private sector in 2004 and even stronger interest in 2008, little tangible results were achieved in terms of public policy or investment. The tragic political turmoil of 2008 diverted attention away from reforms and rehabilitation of public goods. Two, analytical depth needs to be accompanied by an understanding of the political economy. The two ICAs done in Kenya are not unusual in its cursory review of the political dynamics at work in society. The policy advice in most cases does not fully reflect the institutional constraints and resistance that would follow particular reform measures. As a result, most of the recommendations are not sufficiently compelling to stimulate government ownership or action. Three, producing fresh empirical results on economic issues is a powerful way to raise awareness and add credibility to policy dialogue. The data generated by the enterprise surveys and used in the ICAs is often widely used and cited locally, even when the dissemination is restricted. Four, dissemination has value, even if no immediate results are discernible. In the case of Kenya, the 2008 report was more widely disseminated and reached a broader audience than the previous one. In either case, there was little impact on government action. The difference, however, is reflected in the higher awareness of local stakeholders, which could be seen as an impact on public opinion. 95 Ratings Summary 2004 ICA Criterion Rating Comments Results Moderately The Bank used the findings as an input into a 2005 growth report and Satisfactory the IFC also referred to it. The use of the report by the Government was less visible and possibly not significant. The active public- private sector consultations suggest that the awareness of investment climate issues has increased. Strategic Moderately The report was a timely product well aligned with Government relevance Satisfactory development agenda and Bank priorities. The Government‘s ownership, however, was weak. Technical quality Satisfactory The report used the Bank‘s state-of-the-art enterprise survey techniques for capturing private sector (formal manufacturing) views on constraints to business growth. The analyses used appropriate tools for benchmarking including inter-temporal and cross-country comparisons. Recommendations ranged from clear and actionable to general. Dialogue Moderately No evidence could be found of on the dissemination and outreach, Satisfactory other than the recollection of a few local stakeholders. It appears, however, to have reached some of the key targets including private sector advocacy groups. Ratings Summary 2008 ICA Criterion Rating Comments Results Moderately The Bank used the findings as an input in the CAS and a 2008 Unsatisfactory growth report. The use of the report by the Government is not evident and possibly not significant. Public awareness of investment climate issues may have risen, as Government consultations with private firms expanded. The limited intermediate outcomes achieved (e.g., greater access to credit) are not traceable to the report. Private investment and FDI (as a share of GDP) did not increase as envisaged. Strategic Satisfactory The 2008 ICA was a timely product for monitoring the investment relevance climate and for providing input into the 2008 CAS. It is also well aligned with the Government‘s economic recovery program. It did not enjoy sustained interest from the Government, however. Technical quality Satisfactory The report was prepared by a strong team of technical specialists responsible for the survey and the analysis. Appropriate analytical tools were applied to the survey results and cross-country data. Key issued were discussed and presented well. Recommendations, however, ranged from clear and actionable (finance) to very general (taxes). Dialogue Satisfactory There is evidence of direct of efforts spent on dissemination or outreach to a wide audience with good media coverage. The findings were discussed with several government officials and some members of parliament. The report itself has a good summary and useful graphics to convey analytical findings. It appears to have reached the right audiences (e.g., the Kenya Association of Manufacturers) and the dissemination effort was clearly well-targeted. 96 Detailed Assessments: 2004 ICA Note: The tables in each section are not ratings sheets. They are tally sheets that allow the evaluator to tick off key characteristics of each AAA product. The number of ticks in each column may be summed to help inform an overall rating for each product. A. Results Comments Results indicators Did the product have… …clear results/objectives defined Yes. The objectives were stated in the final 2004 ICA report, not in any at inception? previous document. These objectives were analytical and, in line with those of other ICAs, were to assess the current performance of formal manufacturing firms, identify the key constraints on their growth and competitiveness, and to prioritize and assess policy priorities to promote private sector development. The product did not define any results in terms of the use of findings by the country or by the Bank, although the 2004 CAS and the Bank‘s 2005 Growth and Productivity report used those findings (see below). …clearly defined indicators YES/NO. While the product did not provide a clear statement of defined at inception? indicators at inception, the report used (a) age of capital, investment, productivity growth, and exports as indicators of performance of manufacturing firms; and (b) ICA survey responses to identify key constraints to competitiveness and suggest policy priorities. No indicators relating to achievement of analytical objectives, or of results on the client or on the Bank were defined. …a clear strategy to achieve YES IN PART. The product used the standard ICA strategy to achieve results (log-frame)? its analytical objectives, namely the use of a survey to gauge constraints to competitiveness and to prioritize policies. Although the product did not state a clear strategy for the use of those results by the country or the Bank, the 2004 CAS indicates how those results were to be used (see below). To what extent were the expected YES. The expected analytical results were explained to the client. The results and the results-chain Bank‘s result chain was likely to comprise the sequence of the 2004 ICA explained to the client? and the Bank‘s report entitled ―Kenya Growth and Competitiveness‖, for which the ICA served as a key input (see next section on results on the Bank). It is doubtful that the Bank explained to the client any expected results in the country – those expectations are nowhere to be found. The expected operational results on the Bank are obvious enough and may have been explained to the client, although there is no evidence of it. Results achieved In the client country, were the findings used… …in government policy law or Quality of labor: Attempts to improve the Technical and Vocational regulation? Education and Training system have been thus far unsuccessful and not traceable to the 2004 ICA. Access to Credit: Access to credit improved due more to better macroeconomic policies that induced a shift out of government securities and into credit to the private sector and less so to the 2004 ICA recommendations covering private credit bureaus, corporate registries, 97 collateral and property registries, public record systems, and SMEs. While the latter are critical for improving access particularly to SMEs, as indicated in the Bank-IMF 2005 FSAP, progress in these areas was too muted to explain the significant increase in credit access and the decline in interest rates that occurred during the last several years. Corruption: In 2003 Kenya signed and ratified the United Nations Convention Against Corruption. Since 2003 and though 2010, Kenya passed legislation to fight corruption, including the Anti Corruption and Economic Crimes Act (2003), the Public Officers‘ Ethics Act (2003), the Financial Management Act (2004), the Public Procurement and Disposal Act (2005), the Proceeds of Crime and Anti-Money Laundering Act (2009) (TI), and other governance-related reforms under the anti- corruption strategy since April 2005. Specifically, Kenya‘s threshold program aims to reduce opportunities for corruption in public governance and targets corruption in public procurement, the delivery of health care, and the monitoring and evaluation of reforms. More recently, Kenya passed the Proceeds of Crime and Anti-Money Laundering Act (2009) and enacted a new constitution (2010) that aims to reduce corruption by improving checks and balances. Roadblocks, an important source of police corruption, were reduced in number from over 100 to 34 today. Findings from the 2004 ICA, from Transparency International, and from other studies probably contributed to these developments. Apart from the ICA, the Bank sought to influence control of corruption through project implementation and advice to the Internal Audit Department, not through broad policy dialogue. Taxation: Tax reforms in Kenya which shifted revenues from direct to indirect taxation and from international to domestic taxes occurred prior to the 2004 and 2008 ICAs. The Kenya overall tax rate remained at about 50 percent during 2004-2010 (DBI). Accordingly the 2004 ICA did not have an effect in this area. Currently, the Ministry of Finance is conducting a review of taxation, with both IFC and IMF input, not Bank input. Electricity: The Government established its long-term vision and policy framework for the sector in the late 1990s and early 2000s, culminating in the 2004 Energy Policy and the 2006 Energy Act. These were linked to Bank support under its 1997 Kenya Energy Sector Reform Project (that closed in 2004) and not linked in any significant additional way to the 2004 ICA data or its recommendations. The ICA recommendation was to implement that project. Transportation: To increase road rehabilitation maintenance, the Government increased the Road Maintenance Fuel Levy in 2006. The 2004 ICA had recommended raising funding for maintenance, but without the detail necessary to warrant any link to the decision to raise the maintenance levy. To improve the institutional framework, the 2007 Road Bill created three statutory road authorities covering respectively major roads, rural and small town roads, and urban roads. The Bill is not linked in any way to the 2004 ICA recommendations. A key Bank recommendation (included in the 2004 ICA and other documents) was to convert the Port of Mombasa into a landlord port, which has not happened yet. 98 …in design of spending or YES. Transportation: There is no data that would indicate if investment investment programs? programs have increased or improved the road network. However, there is reportedly a $5-6 billion 5-year road investment to reduce the backlog from years of neglect that the findings of the 2004 ICA may have helped motivate. Most investment has focused on road rehabilitation, not new construction. Nevertheless, this investment is not yet reflected into improvements in the overall quality of transport related infrastructure (see ―Monitoring of Impact‖ below). Electricity: Although electricity production increased markedly after 2000, much of it has been from thermal sources recently, particularly after 2006, as increased demand coupled with drought tested the system‘s capacity. The 2004 ICA probably contributed to raise awareness of the power constraint and investment needs by measuring firm perceptions, outages, production lost due to outages, and ownership of generators by firms. …to raise stakeholder awareness / YES. The ICA‘s measurement of well-known problems may have capacity? contributed to raise awareness. Some of these numbers are not available elsewhere and were cited in news and reports. The first Kenya Competitiveness Benchmark Report, published by the Kenya Association of Manufacturers in 2006 cites some of the constraints identified in the 2004 ICA. Google shows 43 results for the 2004 ICA. Nevertheless, in some relevant government quarters (e.g., Kenya Investment Authority), awareness of the 2004 ICA appeared to be weak. Some capacity building may have been contributed through the participation in the study of Kenya‘s Institute for Public Policy Research and Analysis. …to build a coalition for change? YES. The 2004 ICA may have informed private sector coalitions, such as the Kenya Private Sector Alliance (KEPSA), contributing to the quality of public-private sector dialogue (a focus of the ICA recommendations). Prior to 2004, dialogue largely resulted from private lobbying. After 2004, dialogue became more structured, particularly with the creation, in 2003/04 of the National Economic and Social Council, chaired by the President of Kenya and the Prime Minister, and which meets about four times per year. But it would be far-fetched to say that the 2004 ICA played a significant role in building those coalitions. …to influence the donor There is no indication that donors did anything differently as a result of community? the 2004 ICA findings. IFC used the ICA but suggested that its use was limited as it became outdated. In the Bank, were the findings of the product used in …Bank strategy formulation YES. Preparation of the 2004 ICA informed the Bank Group‘s CAS (CAS/PRSP)? issued on May 19, 2004 (i.e., prior to the completion of the final ICA report in November of that year), particularly in developing the CAS‘s objective of reducing the cost of doing business and improving the investment climate, one of its four areas of proposed support. Such support included a FIAS study of the administrative and regulatory costs of doing business in Kenya, with both the ICA and the FIAS study informing a Bank report on Growth and Competitiveness (see next line). Furthermore, in the context of CAS preparation, Kenya‘s Private Sector Alliance met with the Bank Group, including its Bank-IFC-PPIAF private sector development team, and discussed the Bank‘s CAS strategy for private sector-led growth, including a presentation by the Bank of the 2004 ICA. 99 …the design of Bank lending YES. The Bank used the ICA as a key input for the Bank‘s report products or AAAs? entitled ―Kenya Growth and Competitiveness‖ (Report No. 31387-KE, January 27, 2005) If there were recorded results, is YES IN PART. Recorded results on the major PSD constraints were there evidence that they are likely limited to increased access to credit, possibly due to a better to be sustainable? macroeconomic environment that, based on Kenya‘s long-term track record and recent developments carries some risks (noted in the IMF‘s January 2010 Article IV Report) of not being sustained. Progress on corruption is too incipient to merit a significant likelihood of sustainability at this point. Progress on road maintenance investments appears sustainable, given the dedicated funding available from the increased maintenance levy. Progress on power supply capacity has considerable risks of sustainability due to risks of drought and slow progress in diversifying energy sources. Progress on deregulation of licenses appears sustainable, particularly if the law that would ―license‖ licenses is approved. Monitoring of impact Did the Bank or the client The Bank did not assess the product‘s specific impact. IEG rated undertake an assessment of the ―moderately satisfactory‖ the outcomes on ―Reducing the Cost of Doing product‘s impact? Business and improving Investment Climate‖ under its assessment of the 2004 CAS Completion Report. The period covered by the latter included both the 2004 and 2008 ICAs. What intermediate outcomes or Perhaps the most important contribution of the ICA was its measurement indicators reflect the influences of of constraints weighing on business activities. Some of these (e.g., access the ICA? to finance, power outages) can actually be used outcome indicators, although caution is warranted on the subjective nature of many of the indicators. Objective indicators of intermediate outcomes relating to the main factors weighing on the business climate were as follows: Corruption: Transparency International‘s bribery index increased from 2005 to 2008, the last year for which data is available; perceptions of corruption increased through 2006 and show conflicting trends thereafter through 2008. Nevertheless, comparison of the 2003 and 2007 ICA surveys shows a decline in the percentage of manufacturing firms that identified corruption as a major or very severe constraint. The widespread view is that only some lower level officials have been successfully prosecuted. Taxation: The Kenya overall tax rate remained at about 50 percent during 2004-2010 (DBI). Kenya has undertaken reductions in tax rates prior to the period covered by the ICAs Access to Credit: The ratio of credit to the private sector to GDP increased after 2003, after declining since 1993. Crime: The reported overall number of crimes declined after 2004, following increased crime during the previous two years. Informality: The Kenya Bureau of Statistics provides data on the percentage of self-employed and unpaid family workers in total employment for 2003-2007. This percentage increased from 65.7 in 2003 to 67.4 percent in 2007. Electricity: The ICA provides a number of indicators (e.g., share of firms experiencing outages). Comparison of the 2003 and 2007 ICAs that 100 electricity supply outcomes for manufacturing deteriorated between the two years. Transportation: A summary indicator is the Bank‘s Logistics performance index (Quality of trade and transport-related infrastructure rated at 1 to 5). Data are available for 2006 (2.15) and 2009 (2.14) and indicate little if any change. The ICA‘s showed an increase in the share of manufacturing firms identifying transportation as a major or very severe constraint. Nevertheless, a number of the observers interviewed during the mission pointed to improvements in transportation, particularly on road rehabilitation. Private Investment: This is a higher level intermediate outcome indicator reflecting possible influences of the ICA is the pace of private investment, particularly in manufacturing. Total private investment from 2002 to 2008 hovered within 12-16 percent of GDP with no discernible trend. FDI fluctuated from 0.2 to 2.7 percent of GDP, again with no discernible trend. What results / outcomes are The one outcome (increased access to credit) cannot be traced to the 2004 attributable to this ICA? ICA. Other products that have a much stronger claim of attribution include the 2005 FSAP and the IMF Article IV reports and discussions. Discussion of Results The product defined analytical objectives and, by and large, achieved those objectives. These included assessment of the current performance of formal manufacturing firms, identification of the key constraints on their growth and competitiveness, and to prioritization/assessment of policy priorities to promote private sector development. The Bank used the findings as an input into a 2005 growth report and the IFC also used the ICA results in its work. Use of the product by the Government was more muted and possibly not significant as an input into policy developments or investments in the key areas discussed by the ICA. Although difficult to measure, development of private sector fora suggests that awareness of investment climate issues may have increased. Some impact of the ICA on this awareness is likely. However, the limited intermediate outcomes clearly achieved (e.g., greater access to credit) are not traceable to the 20004 ICA. Moreover, private investment and FDI (as a share of GDP) did not increase, so there was no medium or big-bang effect on the investment climate following the ICA. B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. 30 Comments Strategic relevance Was the delivery of the output timely YES. The task was prepared throughout a period of 29 months. Work and on schedule? began in July 2002 with preparation for the survey. The survey was conducted in 2003. The report was delivered to the client on 11/30/2004. Despite a somewhat lengthy preparation period, implementation of the task was timely enough to provide input to the 30 See IEG (2008). Using knowledge to improve development effectiveness: An evaluation of World Bank economic and sector work study. Paragraph 5.21 through 5.29; and Development Results in Middle-Income Countries: An Evaluation of the World Bank‘s Support study page 37. 101 Comments 2004 CAS. Was the issue identified as a development constraint in (note all that apply)… …the relevant CAS? YES. The 1998 CAS targeted Bank support to improving economic governance, a key area identified in the 2004 ICA. The 2004 CAS used the results of the 2004 ICA to formulate a broader strategy for reducing the cost of doing business and improving the investment climate. …previous lending or nonlending YES. Identification of investment climate issues in Kenya already work? had a long-standing history by the time of the 2004 ICA. The World Bank's Regional Program on Enterprise Development (RPED) surveyed firms in 1993, 1994, 1995, and 2003 (the survey for the 2004 ICA) in Kenya to investigate whether the private sector was growing and whether government policies were having a positive impact on firm productivity. Furthermore, the UN (UNIDO) together with Centre for the Study of African Economies, the Universities of Oxford Göteborg, and several Kenyan collaborators conducted a survey in 2000 that was consistent with the Bank‘s 1995 RPED Survey. …policy dialogue with clients? YES. A number of opportunities for Bank-Government policy dialogue on the investment climate may have emerged in preparation for the ICA. Perhaps the most significant was preparation and discussion of the Bank‘s 2003 report entitled ―Kenya: A Policy Agenda to Restore Growth‖ (Report No. 25840-KE). This report included chapters on the manufacturing sector and on the investment climate. Together with discussion of other previous firm surveys and reports, preparation and discussion of the 2003 report with the Government may have furnished guidance on constraints to be investigated/analyzed by the 2004 ICA. …donor coordination forums? The Bank plays a leadership donor coordination role in Kenya, through the Country Director‘s chairing of the Kenya Donor Consultative Group monthly meetings. These are more likely to use findings as inputs for donor support more than identifying new constraints. The Joint Statement by Development Partners at the Kenya Consultative Group Meeting of 24th– 25th November 2003 (Nairobi) made no reference to investment climate issues. In cases where the dialogue is difficult, There is nothing to suggest that dialogue was difficult. did the product, did the task find a solution? Ownership Is the product part of an overall Although the authorities most likely agreed to it (there is no record of program of AAA to which the such agreement), this product was not part of the planned set of AAA authorities have contributed or agreed? outputs under the 1998 CAS. Most of the work was completed just before the 2004 CAS (2004) and was an input to that CAS. Did the client request or commission The product was not the result of a commission by the Government of the specific product? Kenya. It was conducted under the Bank‘s Regional Program for Enterprise Development. 102 Comments Did the client cover some or all of the The costs of the product were covered by the Bank (88 percent) and a costs? Bank-executed Swedish Trust Fund (12 percent). Did key decision makers collaborate YES. The Kenya Ministry of Trade and Industry, the counterpart for with, discuss or provide feedback on the product, provided input. the product? Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? YES. The Kenya Institute for Public Policy Research and Analysis (KIPPRA) in Nairobi conducted the survey. …analyze the results and write the The results were analyzed and the report was written entirely by Bank report? staff. …provide peer review or comments YES. Insights and analysis were provided by the European Union, on the draft report? the Public-Private Infrastructure Advisory Facility, the Africa Project Development Facility, the Kenya Investment Promotion Center, the University of Nairobi Institute for Development Studies, DFID, the Kenya Private Sector Alliance, IFC, UNDP, Market Intelligence (a Nairobi-based finance and business journal), Transparency International, and the African center for Economic Growth. …organize workshops or discussions The Bank‘s report Kenya Growth and Competitiveness was discussed about the findings? at a workshop held in Nairobi on September 20, 2004. As indicated, this report relied extensively on the 2004 ICA findings. The workshop was organized by the Ministry of Trade and Industry (the Government counterpart for the 2004 ICA) and co-sponsored by DFID. Discussion of Strategic relevance and ownership The 2004 ICA was a timely product for measuring and analyzing constraints to business, informing the 2004 CAS, and providing input into the Bank‘s 2005 growth report. It addressed a well-identified development constraint, namely the long-standing low international ranking of Kenya‘s investment climate. Its findings and recommendations were potentially useful for both the government and non-government organizations, including other donors. Government engagement was muted on the ICA, but possibly stronger on the 2005 report to which the ICA provided input. The design, which a) coupled survey results with other indicators to identify key constraints with Bank sector knowledge to focus recommendations and b) was to be used in the a broader Bank 2005 growth report, was appropriate. The survey was limited to formal manufacturing firms. C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. YES. The 2004 ICA included the use of the Bank‘s long- standing make use of current and relevant ICA experience. knowledge available)? …cite relevant examples of practice or YES. The 2004 ICA includes some appropriate cross-country research from other countries? comparisons that use other country ICAs. 103 …discuss the specific institutional and YES IN PART. The 2004 ICA includes a policy matrix covering policy context for the issue? the main constraints that it identified. The ICA however has little or no discussion of the institutional and policy context. The follow-up ―Kenya Growth and Competitiveness‖ discusses context more broadly, based on findings from the ICA and other sources. …collect and analyze existing local data? YES. The 2004 ICA collected data to fulfill its main analytical objective. …generate new evidence? YES. The 2004 ICA contributed with new subjective and objective new evidence in a number of areas (e.g., electricity, capital equipment). …include recommendations? YES. Following standard ICA practice, the 2004 ICA provided a matrix of policy recommendations. These covered five key areas: quality of labor, access to finance, corruption and crime, infrastructure, and private-public sector dialogue. Is the policy advice consistent with the YES. The policy advice focuses on quality of labor, access to preservation of social order? finance, corruption and crime, infrastructure, and private-public sector dialogue. Improvements in all areas would be tantamount to a better and more stable social order. Do the recommendations include specific YES. Many recommendations have a fair degree of specificity. actions to be taken by specific actors? Specificity of actors seldom goes beyond reference to the Government of Kenya. Was the task team staffed with the YES. The team included Christopher Blattman (MPA Harvard necessary expertise? Kennedy); Linda Cotton (SAIS Johns Hopkins Graduate working on Africa ICAs); Vyjayanti Desai (PSD Specialist, Public Administration and Finance); Ibrahim Elbadawi ( a seasoned economist with research experience); James Habyarimana, Jean Michel Marchat, Vijaya Ramachandran, Manju Kedia Shah, Peter Kimyu, Rose Ngugi, Arne Bigsten, and Mans Soderbom. Did the product receive adequate YES. Makhtar Diop (Country Director) and Axel Peuker managerial attention? (manager, Investment Climate) provided significant managerial and substantive attention. Mr. Diop‘s strong engagement continued through preparation of the 2005 Kenya Growth and Competitiveness report, to which the 2004 ICA provided significant input. Did the product receive a sufficient Yes ($135,000). budget? Review of contents Was the draft peer-reviewed by appropriate YES. Michael Fuchs (financial sector expert, currently adviser at experts? AFTFE) and Andrew Stone (with long-standing ICA expertise, currently Lead Private Sector Development Specialist at MNSFP (MNA)) provided a number of suggestions and comments. Other comments were provided by Praveen Kumar and Christiane Kraus (Sr. Economists at the time). Were the peer-review comments taken into There is no record of the peer reviews. account? Was the feedback from the client There is no record of feedback from the client (Kenya Ministry of incorporated? Trade and Industry). 104 Discussion of Technical quality The thrust of the 2004 ICA is (a) that the competitiveness disadvantage of Kenya‘s manufacturing firms vis-à- vis strategic competitors like China and India is due to outdated plant and equipment and comparatively costly labor; and (b) that productivity growth has been zero or negative. Outdated capital resulted from decades of low investment levels. Excessive labor costs may have resulted from increases in minimum and public sector wages. In addition to high labor costs, high levels of crime and corruption and dilapidated infrastructure were main contributors to an adverse investment climate. The 2004 ICA used the Bank‘s well-developed state-of-the-art investment climate survey techniques for gauging private sector (formal manufacturing) views on constraints to business development and developed various analyses of those constraints. The analyses included a) an overview of macro trends and prospects as well as a general discussion of some key areas with a bearing on PSD (education/health, financial sector, trade, crime); b) more detailed analysis of the capital stock (including its age), investment, productivity growth, and manufacturing exports, as key indicators of manufacturing sector performance; c) an analysis of the competitiveness (i.e. productivity vs. costs) of labor and finance; d) discussion of some of the key constraints to competitiveness (corruption, crime, and infrastructure), with passing reference to a few others; and e) discussion of policies to improve the quality of labor and access to finance as well as to address corruption, improve the legal framework, and remove barriers to investment. The analyses used regression tools (firm level productivity, labor earnings), benchmarking through cross-country comparisons (corruption, access to finance, ratio of wages to manufacturing value added), benchmarking through cross-regional comparisons (ICA constraints), in-country comparisons of time trends of related variables (e.g., wage versus. productivity growth). The report has some references to political economy issues and it measures a political economy dimension (political instability as a constraint). Recommendations range from clear and actionable to general and/or empty. Presentation of the findings was adequate. D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local language? Yes (English) …made available on a website? YES. The report is available in the Bank‘s external website. …discussed with senior policy-makers? YES. There is no record of discussion of the ICA with senior policy-makers. …presented at a workshop, conference or YES. While there is no record of a presentation of the ICA at a on the web? workshop or conference, the concurrent 2005 Bank report entitled Kenya growth and Competitiveness, which drew upon the results of the ICA, was discussed at a workshop in Nairobi in September 2004. The workshop was organized by the Ministry of Trade and Industry and attended by representatives of several ministries, government agencies, private sector and donors. …covered in the media? There is no surviving evidence of media coverage. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? YES. Data from the 2004 ICA was used from time to time in policy discussions, either directly or through other Bank work. 105 …discussions/contact with stakeholders? YES. There is direct evidence that the ICA was used for contact with stakeholders during the initial dissemination of the product. …lending and non-lending products? YES. The product helped sustain dialogue by providing input into the Bank‘s 2005 report entitled Kenya Growth and Competitiveness. …other means (please discuss below)? Discussion of Dissemination and sustained dialogue No evidence could be found of any effort spent on dissemination or of outreach to a wide audience with good media coverage, although general recollection of the ICAs suggests that some effort must have been deployed.. The report itself had an adequate summary and exhibited useful graphics to convey its analytical findings. It appears to have reached the right audiences (e.g., the Kenya Association of Manufacturers) but there is no evidence of effort at appropriate targeting. 106 Detailed Assessments: 2008 ICA Note: The tables in each section are not ratings sheets. They are tally sheets that allow the evaluator to tick off key characteristics of each AAA product. The number of ticks in each column may be summed to help inform an overall rating for each product. A. Results Comments Results indicators Did the product have… …clear results/objectives defined at YES. The concept note stated that the objective of the 2008 ICA inception? was to evaluate the investment climate in all its operational dimensions and indicated that the ICA was to provide a ―crucial diagnostic‖ of the business environment to the Government, to examine changing patterns over time (given the availability of the previous 2003 survey), and to benchmark Kenya‘s competitiveness against other countries. The Bank used the ICA findings as in input to the Bank‘s report entitled ―Kenya Accelerating and Sustaining Inclusive Growth‖. …clearly defined indicators defined at YES IN PART. The concept paper outlined the parts of the ICA inception? survey questionnaire as well as a series of objective indicators that would be used to validate the perceptions emerging from the survey. The ICA used (a) productivity and unit labor costs to assess performance; and (b) ICA survey responses to identify key constraints to competitiveness and suggest policy priorities. No indicators relating to achievement of analytical objectives, or of results on the client or on the Bank were defined. …a clear strategy to achieve results YES. The product used the standard ICA strategy to achieve its (log-frame)? analytical objectives, namely the use of a survey to gauge constraints to competitiveness and to prioritize policies. Although the product did not state a clear strategy for the use of those results by the country or the Bank, the 2008 CAS used the results to design its support for improvements of Kenya‘s business environment and competitiveness (see below). To what extent were the expected results YES IN PART. The expected analytical results were explained to and the results-chain explained to the the client. The Bank‘s result chain was likely to comprise the client? sequence of the 2008 ICA and the Bank‘s report entitled ―Kenya Accelerating and Sustaining Inclusive Growth‖, for which the ICA served as a key input (see next section on results on the Bank). It is doubtful that the Bank explained to the client any expected results in the country – those expectations are nowhere to be found. The expected operational results on the Bank are obvious enough and may have been explained to the client, although there is no evidence of it. Results achieved In the client country, were the findings used… …in government policy law or Access to Credit: Access to credit improved due more to better regulation? macroeconomic policies that induced a shift out of government securities and into credit to the private sector and less so to the 2008 ICA recommendations that, as the 2004 ICA, covered private credit 107 bureaus, corporate registries, collateral and property registries, public record systems, and SMEs. While the latter are critical for improving access particularly to SMEs, as indicated in the Bank- IMF 2005 FSAP, progress in these areas was too muted to explain the significant increase in credit access and the decline in interest rates that occurred during the last several years. Corruption: Kenya passed the Proceeds of Crime and Anti-Money Laundering Act (2009) and enacted a new constitution (2010) that aims to reduce corruption by improving checks and balances. Findings from the 2004 and 20008 ICAs, from Transparency International, and from other studies probably contributed to these developments. Apart from the ICAs, the Bank sought to influence control of corruption through project implementation and advice to the Internal Audit Department, not through broad policy dialogue. Taxation: Tax reforms in Kenya which shifted revenues from direct to indirect taxation and from international to domestic taxes occurred prior to the 2004 and 2008 ICAs. The Kenya overall tax rate remained at about 50 percent during 2004-2010 (DBI). Accordingly the 2008 ICA did not have an effect in this area. Currently, the Ministry of Finance is conducting a review of taxation, with both IFC and IMF input. Electricity: The Government established its long-term vision and policy framework for the sector in the late 1990s and early 2000s, culminating in the 2004 Energy Policy and the 2006 Energy Act. These of course predated the 2008 ICA. The latter‘s recommendations to establish clear rules for private generators open access the transmission network and to establish and a pricing policy that maintains the financial viability of power companies, while appropriate, were general and unlikely to merit attention or attribution in any policy developments in this area. Transportation: A key Bank recommendation included in the 2004 and 2008 ICAs and other documents was to convert the Port of Mombasa into a landlord port, which has not happened yet. Similarly, attempts to secure significant private sector participation in port operations, another 2004 and 2008 ICAs recommendation, did not materialize. Licensing: The 2008 ICA documented the importance of licensing as a constraint on the investment climate and measured the time managers spent on it. However, impact on policy after 2004, including the reduction in the number of licenses, simplification of some of those that remained in place and the submission to Parliament of a Business Regulation Bill under which a Government body would ―license‘ licenses, may be attributable to a larger extent to FIAS advice (under a regulatory study that covered taxation, licensing, infrastructure, customs clearance, land reform, and security and corruption) than to the 2008 ICA (or, for that matter, to the 2004 ICA, which covered licensing only cursorily. Nevertheless, FIAS acknowledges the ICAs work in measuring licensing as a perceived constraint and as an activity on which managers spend significant amounts of time. 108 …in design of spending or investment YES IN PART. Transportation: There is no data that would programs? indicate if investment programs have increased or improved the road network. However, there is reportedly a $5-6 billion 5-year road investment to reduce the backlog from years of neglect that the findings of the 2008 ICA may have helped reinforce. Most investment has focused on road rehabilitation and maintenance, not new construction. Nevertheless, this investment is yet to be reflected into improvements in the overall quality of transport related infrastructure (see ―Monitoring of Impact‖ below). Electricity: Although electricity production increased markedly after 2000, much of it has been from thermal sources recently, particularly after 2006, as increased demand coupled with drought tested the system‘s capacity. The 2008 ICA, as its former sibling, probably contributed to motivate investment by reinforcing awareness of the power constraint and investment needs through updating firm perceptions, outages, production lost due to outages, and ownership of generators by firms. …to raise stakeholder awareness / YES. The ICA‘s measurement of well-known problems may have capacity? contributed to raise awareness. Some of these numbers are not available elsewhere and were cited in news and reports. The second Kenya Competitiveness Benchmark Report, published by the Kenya Association of Manufacturers in 2008 cites some of the constraints identified in the 2008 ICA. Google shows 90 results for the 2008 ICA. Nevertheless, in some relevant government quarters (e.g., Kenya Investment Authority), awareness of the 2008 ICA appeared to be weak. …to build a coalition for change? The 2008 ICA may have further informed private sector coalitions, such as the Private Sector Alliance, as well as National Economic and Social Council, established years earlier as an arrangement for private-public sector dialogue to result in recommendations to government (which admittedly has resulted in little action). But it would be far-fetched to say that they played a significant role in building those coalitions or allowing for that dialogue. …to influence the donor community? YES. DFID and the EU have been supporting private sector development, particularly in the context of helping the government develop its follow-up private sector strategy to the initial 2006-2010 version. Their efforts rely to some extent relied on the information provided by the 2008 ICA, with 75 percent the latter financed by the DFID Trust Fund. IFC has used the ICA but suggested that its use was limited as it became outdated. In the Bank, were the findings of the product used in …Bank strategy formulation YES. The ICA informed the 2010 CAS‘s proposed strategy to (CAS/PRSP)? improve Kenya‘s investment environment and competitiveness, which included support for i) reforms that would reduce the cost of doing business, strengthen the financial sector, and improve logistics, trade across borders, and regional integration; ii) road sector rehabilitation, institutional strengthening and private sector financing; iii) expansion of private participation in the Port of Mombasa; iv) expansion of the Nairobi Airport towards its becoming an aviation hub; v) restructuring of the railway concession; vi) resource mobilization to expand electricity generation and strengthening of power sector institutions; and vii) 109 developing a new water supply for Nairobi, clarifying responsibilities within the water sector, and identifying private financing for small municipal water systems. CAS Investment Climate objectives also covered agricultural issues (expansion of irrigation and enhancement of agricultural productivity) that were not within the purview of the 2008 ICA. …the design of Bank lending products YES. As with the 2004 ICA, the 2008 ICA informed a new growth or AAAs? report, in this case the Bank‘s 2008 report entitled ―Kenya Accelerating and Sustaining Inclusive Growth‖ (Report No. 42844- KE). Both the 2004 and 2008 ICAs measured the strong importance of security concerns in business perceptions of constraints to investment and helped motivate the Bank‘s 2009 report entitled ―Kenya Economic Development, Police Oversight, and Accountability Linkages and Reform Issues‖ (Report No. 44515-KE). The ICA may have also played a motivating role in a number of lending operations, most likely on the 2009 ―Kenya - Electricity Expansion Project‖ (P103037) If there were recorded results, is there Recorded results on the major PSD constraints were limited to evidence that they are likely to be increased access to credit, possibly due to a better macroeconomic sustainable? environment that, based on Kenya‘s long-term track record and recent developments carries some risks (noted in the IMF‘s January 2010 Article IV Report) of not being sustained. Progress on corruption is too incipient to merit a significant likelihood of sustainability at this point. Progress on road maintenance investments appears sustainable, given the dedicated funding available from the increased maintenance levy. Progress on power supply capacity has considerable risks of sustainability due to risks of drought and slow progress in diversifying energy sources. Progress on deregulation of licenses appears sustainable, particularly if the law that would ―license‖ licenses is approved. But all these are only very weakly attributable to the ICA. Monitoring of impact Did the Bank or the client undertake an The Bank did not assess the product‘s specific impact. IEG rated assessment of the product‘s impact? ―moderately satisfactory‖ the outcomes on ―Reducing the Cost of Doing Business and improving Investment Climate‖ under its assessment of the 2004 CAS Completion Report. The period covered by the latter included both the 2004 and 2008 ICAs. What intermediate outcomes or Corruption: TI‘s Bribery index (www.tikenya.org) is available indicators reflect the influences of the only up to 2008. Kenya‘s rank in TI Corruption Perceptions Index ICA? dropped seven places since 2009. The widespread view is that only some lower level officials have been successfully prosecuted. Taxation: The Kenya overall tax rate remained at about 50 percent during 2004-2010 (DBI). Access to Credit: The ratio of credit to the private sector to GDP increased after 2003, after declining since 1993. Data is available only through 2008. There is a sense that financial markets have increased considerably in sophistication. Crime: The reported overall number of crimes declined after 2004, following increase crime during the previous two years. Data is available only through 2008. There is a sense that the security business is booming. 110 Informality: The Kenya Bureau of Statistics provides data on the percentage of self-employed and unpaid family workers in total employment for 2003-2007. This percentage increased from 65.7 in 2003 to 67.4 percent in 2007. Electricity: The ICA provides a number of indicators (e.g., share of firms experiences outages). Comparison of the 2003 and 2007 ICAs that electricity supply outcomes for manufacturing deteriorated between the two years. Data is unavailable after 2007. Transportation: A summary indicator is the Bank‘s Logistics performance index (Quality of trade and transport-related infrastructure rated at 1 to 5). Data are available for 2006 (2.15) and 2009 (2.14) and indicate little if any change. The ICA‘s showed an increase in the share of manufacturing firms identifying transportation as a major or very severe constraint. Congestion in Nairobi is thought to be on the rise. Nevertheless, a number of the observers interviewed during the mission pointed to improvements in transportation, including rehabilitated roads and full automation in the Port of Mumbassa. Air transport is on the rise. Private Investment: This is a higher level intermediate outcome indicator reflecting possible influences of the ICA is the pace of private investment, particularly in manufacturing. Total private investment from 2002 to 2008 hovered within 12-16 percent of GDP with no discernible trend. FDI fluctuated from 0.2 to 2.7 percent of GDP, again with no discernible trend and remained within that range in 2009. Licensing: The 2008 ICA analysis covered licensing as a constraint, although the share of firms indicating it as a constraint was significantly smaller than for the other constraint‘s outlined above. The 2008 ICA analysis followed an by a Government Working Committee on Regulatory Reform established in 2005. It actually counted the licenses (1325). The review led to an elimination of 315 licenses and the simplification of 379 licenses. Furthermore, the Government submitted to parliament. FIAS has provided technical assistance and policy advice in this area. As a result of the efforts, the 2008 ICA documented a decline in the time spend by firm managers on licenses from 14 percent in 2003 to seven percent in 2007. What results / outcomes are attributable The one clear outcome (increased access to credit) cannot be traced to this ICA? to 2008 ICA recommendations. Other products that have a much stronger claim of attribution include the 2005 FSAP and the IMF Article IV reports and discussions. Comparison of the 2004 and 2008 ICAs corroborated the increased access to credit suggested by macro-financial indicators. However, this monitoring contribution is rather removed from the forces to which the results are attributable (e.g., macroeconomic stabilization, the 2005 FSAP, and article IV discussions. 111 Discussion of Results The product defined analytical objectives and, by and large, achieved those objectives. These included evaluation of the investment climate in all its operational dimensions, diagnosis of the business environment for the Government, examination of changing patterns over time (given the availability of the previous 2003 survey), and benchmarking of Kenya‘s competitiveness against other countries. The Bank used the findings as an input into a 2008 growth report. Use of the product by the Government was more muted and possibly not significant as an input into policy developments or investments in the key areas discussed by the ICA. Although difficult to measure, development of private sector fora suggests that awareness of investment climate issues may have increased. Some impact of the 2008 ICA on reinforcing this awareness is likely. However, the limited intermediate outcomes clearly achieved (e.g., greater access to credit) are not traceable to the 20004 ICA. Moreover, private investment and FDI (as a share of GDP) trends did not increase after 2007, so there was no medium or big-bang effect on the investment climate following the ICA. B. Strategic relevance and ownership This section aims to distinguish an ICA that has a clear goal and usefulness from supply-driven ones. 31 Comments Strategic relevance Was the delivery of the output timely YES. The task was prepared throughout a period of 23 months. and on schedule? Work began in January 2007. The survey was conducted in May- July 2007. The report was delivered to the client on 11/30/2004. Implementation of the task was timely enough to provide input to the 2008 Kenya Accelerating and Sustaining Inclusive Growth report and for preparation of the 2010 CAS. Was the issue identified as a development constraint in (note all that apply)… …the relevant CAS? The CAS used the 2004 ICA as an input to formulate its proposed support for improvements in the investment climate. Similarly, the 2010 CAS used the 2008 ICA to formulate its growth objective. …previous lending or nonlending YES. Identification of investment climate issues in Kenya already work? had a long-standing history by the time of the 2008 ICA. The World Bank's Regional Program on Enterprise Development (RPED) surveyed firms in 1993, 1994, 1995, 2003 and 2007 (the survey for the 2008 ICA) in Kenya to investigate whether the private sector was growing and whether government policies were having a positive impact on firm productivity. Furthermore, the UN (UNIDO) together with Centre for the Study of African Economies, the Universities of Oxford Göteborg, and several Kenyan collaborators conducted a survey in 2000 that was consistent with the Bank‘s 1995 RPED Survey. 31 See IEG (2008). Using knowledge to improve development effectiveness: An evaluation of World Bank economic and sector work study. Paragraph 5.21 through 5.29; and Development Results in Middle-Income Countries: An Evaluation of the World Bank‘s Support study page 37. 112 Comments …policy dialogue with clients? YES. With a sounder footing on investment climate issues as the 2008 ICA and CAS were being prepared, it is likely that dialogue on investment climate issues picked up its pace. The most significant other opportunities for such dialogue included preparation of the 2004 Kenya - Micro, Small and Medium Enterprise Competitiveness Project (P085007); the 2004 Kenya - Financial and Legal Sector Technical Assistance Project (P083250); the 2005 ―Kenya Growth and Competitiveness report‖ (Report No. 31387-KE); and the 2007 ―Kenya Unleashing the Potential for Trade and Growth‖ (Report No. 37688-KE). …donor coordination forums? The Bank plays a leadership donor coordination role in Kenya, through the Country Director‘s chairing of the Kenya Donor Consultative Group monthly meetings. These are more likely to use ICA findings as inputs for donor support more than identifying new constraints. There was some discussion of investment climate issues in the Consultative Group Meeting held in Nairobi on April 11-12, 2005. In cases where the dialogue is difficult, There is nothing to suggest that dialogue was difficult. did the product, did the task find a solution? Ownership Is the product part of an overall program Although the authorities most likely agreed to it (there is no record of AAA to which the authorities have of such agreement), this product was not part of the planned set of contributed or agreed? AAA outputs under the 2004 CAS. Did the client request or commission the The product was not the result of a commission by the Government specific product? of Kenya. It was conducted under the Bank‘s Regional Program for Enterprise Development. Did the client cover some or all of the The costs of the product were covered by the Bank (25 percent) and costs? DFID (75 percent). Did key decision makers collaborate Several government officials and members of parliament provided with, discuss or provide feedback on the feedback on the report upon a Bank presentation in June 2008. product? Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? The survey was conducted by the Etude Economique Conseil (Canada) form May 2007 to July 2007. …analyze the results and write the The results were analyzed and the report was written entirely by report? Bank staff. …provide peer review or comments on YES. The Kenya Institute for Public Policy Research and the draft report? Analysis (KIPPRA) and the National Association for Technology Transfer and Entrepreneurial Training provided comments. …organize workshops or discussions Presentation of the findings was conducted through individual about the findings? discussions and a Bank-sponsored event held on June 23, 2008. 113 Discussion of Strategic relevance and ownership The 2008 ICA was a timely product for updating the monitoring of constraints to business, following the 2004 ICA and for providing input into the 2008 CAS and the 2008 growth report. It addressed a well-identified development constraint, namely the long-standing low international ranking of Kenya‘s investment climate. Its findings and recommendations were potentially useful for both the government and non-government organizations, including other donors. Government and non-government engagement was strong in providing feedback on the report. The design, which a) coupled survey results with other indicators to identify key constraints with Bank sector knowledge to focus recommendations and b) was to be used in a broader Bank 2008 growth report, was appropriate. The design was somewhat broader (and more costly) than that of the 2004 ICA, in that it had a wider coverage (included services in addition to manufacturing, and microfirms in addition to formal establishments). C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. YES. The 2008 ICA included the use of the Bank‘s long- standing make use of current and relevant ICA experience. knowledge available)? …cite relevant examples of practice or YES. The 2008 ICA includes some appropriate cross-country research from other countries? comparisons that use other country ICAs. …discuss the specific institutional and YES IN PART. The 2008 ICA includes a policy matrix covering the policy context for the issue? main constraints that it identified. The ICA however has little or no discussion of the institutional and policy context. The follow-up ―Kenya Accelerating and Sustaining Inclusive Growth‖ discusses context more broadly, based on findings from the ICA and other sources. …collect and analyze existing local YES. The 2008 ICA collected data to fulfill its main analytical data? objective. …generate new evidence? YES. The 2008 ICA contributed with new subjective and objective new evidence in a number of areas (e.g., transportation costs, costs from crime). …include recommendations? YES. Following standard ICA practice, the 2008 ICA provided a matrix of policy recommendations. These covered four key areas: access to finance, corruption and crime, infrastructure, taxes and licensing. Is the policy advice consistent with the YES. The policy advice focuses on access to finance, corruption and preservation of social order? crime, infrastructure, taxation, and licensing arrangements. Improvements in all areas would be tantamount to a better and more stable social order. Do the recommendations include YES. Many recommendations have a fair degree of specificity. specific actions to be taken by specific Specificity of actors seldom goes beyond reference to the actors? Government of Kenya. 114 Was the task team staffed with the YES. Main contributors included Giussepe Iarosi (an acknowledged necessary expertise? Bank leader in ICAs); Leonardo Garrido (with training and experience on Public Administration); Ricardo Gonçalves (Finance); Manju Kedia Shah (Survey); Sofia Silva, and Mans Soderbom (Survey). Did the product receive adequate YES. The task‘s documentary record suggests adequate managerial managerial attention? attention (e.g., managerial presence in review meetings and presentation by management to the Government). Did the product receive a sufficient Yes ($412,000). This was probably more than enough, although the budget? higher budget (compared with the 2004 ICA‘s budget) is explained by the broader coverage of the survey. Review of contents Was the draft peer-reviewed by Qmiao Fan (lead Economist), Najy Benhassine (Senior Economist), appropriate experts? Moses Ikiara (ED of the Kenya Institute for Public Research and Analysis, KIPPRA), and Joseph Ngumi (Association of Technology Transfer and Entrepreneurship Development (NATTET)) were the peer reviewers. In addition, Lars Nikolai Grava (Regulatory Simplification (CICRS) of FIAS) and Michael Jarvis (Senior Private sector Development Specialist) also provided written comments. Were the peer-review comments taken The minutes of the meeting concluded on the comments to be taken into account? into account, covering a redrafting of the executive summary, more selectivity and context into the recommendations, a balancing of energy and transport as constraints, and an elaboration of the discussion of skilled workforce constraints. These were taken into account in the final draft. Was the feedback from the client Several government officials and members of parliament provided incorporated? feedback on the report upon a Bank presentation in June 2008. Some of the feedback was incorporated into the final document. Discussion of Technical quality The thrust of the 2008 ICA is that low productivity of Kenyan firms is partly attributed to shortcomings in the investment climate. The report indicates that the productivity of Kenyan firms improved during the 2003-07 period (following stagnant productivity during 1999-03) due primarily to increased capacity utilization and that it was below the productivity of firms in middle income countries in sub-saharan Africa. It further indicates that Kenya‘s adverse business climate, particularly with regard to corruption and deficient power and transportation infrastructure, contributed to drive poor productivity performance. To substantiate these findings, the 2008 ICA used the Bank‘s well-developed state-of-the-art investment climate survey techniques for gauging private sector (formal and informal urban manufacturing and services) perceptions on constraints to business development as well as objective indicators to confirm or reject the perceptions. The analyses included a) an overview of total and private GDP trends; growth and macro trends and prospects as well as a general discussion of some key areas with a bearing on PSD (education/health, financial sector, trade, crime); b) analyses of changes in total factor productivity, including the effect of capacity utilization as an explanatory variable and an analysis of pooled data covering several countries; c) an analysis of the competitiveness (i.e. productivity vs. costs) of labor; d) discussion of some of the key constraints to competitiveness (tax rates, corruption, and electricity/transportation/customs infrastructure, crime, and business licensing) identified through the survey, and a broader discussion and cross-country benchmarking of access to finance, labor markets and human capital, and microenterprises; and e) a policy matrix covering policies aimed at improving the business climate in the key areas identified by the report. The analyses used 115 regression tools (firm level productivity,), benchmarking through the indicated cross-country comparisons (tax rates, corruption, access to finance,), benchmarking through cross-regional comparisons (ICA constraints), in- country time trends (e.g., annual cost of borrowing). The analysis in report makes little or no reference to political economy issues, but it measures a political economy dimension (political instability as a constraint). Recommendations ranged from clear and actionable (finance) to general and/or empty (taxes). An edited version of the final report benefited the presentation of findings. D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local language? Yes (English) …made available on a website? YES. The report is available in the Bank‘s external website. …discussed with senior policy-makers? YES. The report was discussed with several government officials and members of parliament during a Bank June 2008 mission by the Task Manager. …presented at a workshop, conference or YES. The Bank organized an event in June 2008 that was widely on the web? attended, including leaders from the private sector, government civil society, and media and development agencies. …covered in the media? YES. The June 2008 event organized by the Bank received ample media coverage. Sustained dialogue Did the product help sustain engagement through… …policy dialogue? YES. Data from the 2004 ICA was used from time to time in policy discussions, either directly or through other Bank work. …discussions/contact with stakeholders? YES. There is no direct surviving evidence of the use of the 2004 ICA in discussions with stakeholders. Nevertheless, the fairly broad recollection among stakeholders of the ICA suggest that the product may indeed have helped inform those discussions. …lending and non-lending products? YES. The product helped sustain dialogue by providing input into the Bank‘s 2008 report entitled Kenya Accelerating and Sustaining Inclusive Growth. …other means (please discuss below)? Discussion of Dissemination and sustained dialogue There is direct of efforts spent on dissemination or of outreach to a wide audience with good media coverage. The report itself had a good summary and exhibited useful graphics to convey its analytical findings. It appears to have reached the right audiences (e.g., the Kenya Association of Manufacturers) and the dissemination effort was clearly well-targeted. 117 ANNEX 5: VIETNAM Performance Assessment Report on Investment Climate Vietnam Investment Climate Assessment 2006 Vice President Mr. Jemal-ud-din Kassum, Mr. James Adams Country Director Mr. Klaus Rohland Sector Director Mr. Homi Kharas Sector Manager Khalid Mirza Task Leader Amanda Carlier and Noritaka Akamatsu Project ID/ Type/Cost P088218/ESW /$196,000 Number of: Volumes/ Pages 1 volume/ 164 pages Languages English / Vietnamese Document Type Private Sector Development Topics Investment Climate Assessment Unit Poverty Reduction and Economic Management Unit - EASFP Background The case of Vietnam stands out from the others in that it involved a single 2006 ICA, and one that was never published. The report was initiated at a time when the Vietnamese economy was rapidly opening up, evolving from a state-directed model towards a market economy, and when there was a healthy momentum for reform measures to address issues in the investment climate. The ICA‘s focus on the disparity – on commercial terms -- between state-owned firms and private businesses was highly relevant at the time. The reform initiative to amend the legal framework was already underway, and it led ultimately to the new Unified Enterprise Law and the Common Investment Law, which effectively addressed the issue. Bank staff, including the ICA task team, was involved as advisor on these topics to the Prime Minister‘s Research Advisory Group. The findings of the enterprise survey and policy briefs were provided as inputs for the Government, but the ICA and the task team were not responsible for igniting the reform or shaping the contents. The ICA suffered additional setbacks through the turnover of two task managers, within a one year, before it was finished and presented to the Government. The resulting delay of almost two years was highly unfortunate in that the new legislation became effective in the interim. The report missed a crucial opportunity to make a decisive impact on the outcome of the reform and to be seen as such. Instead, when the final report was delivered to the Government, it was already dated and irrelevant. Nonetheless, before the final report was completed, some of the findings were presented as a Chapter in the 2006 Vietnam Development Report (VDR), an annual multi-donor publication which was presented at the Consultative Group meeting. Overall Assessment The ICA has had a modest impact. On results, there is partial evidence that inputs were provided to the legal reform committee and that the interaction with the ICA team helped the government in building the capacity for conducting its own surveys. In addition, some of the findings were used in the Bank‘s Vietnam Development Report. The relevance of the work also diminished greatly over time, owing primarily to the delay. The ownership by the Government was also constrained by the limited engagement of the task team. The technical quality was good on the analytical side, although the recommendations were not fully developed or discussed. By the time the final report reached the Government, it was already dated and not appropriate for publication. The dissemination was unusually narrow, as it reached only the main counterpart in the Government (MPI). 118 Lessons Political economy. The report dealt very openly with topics that could be perceived as sensitive, including the unfair advantages of state-owned enterprises, misguided regulations and corruption. But it offered no guidance concerning the impact of the reforms on the winners and losers or their political influences. PREM/FPD collaboration proved to be a major issue in this case. PREM was critical of the approach and timeliness of the product which was under the FPD jurisdiction. Working relations between staff of the two Networks left much to be desired. Dialogue. Like other cases, the engagement occurred mainly at the beginning and the end of the process. As this process was lengthy, the counterpart went through a long period without interacting with the task team – not a good way to foster ownership. In addition, the engagement could have involved broader collaboration. The ICA covers a variety issues with many agencies in charge. Reaching out to all the key players would have raised awareness and deepened the ownership. Ratings Summary Criterion Rating Comments Results Unsatisfactory The efficacy is modest and confined to the intermediate outputs in the form of Bank reports or advice to the Government. Strategic relevance Moderately The relevance was high, but ICA was overshadowed by Satisfactory broader efforts made toward WTO accession. Technical quality Moderately The report provided good analytical results, based on domestic Satisfactory and international data. Policy advice, however, was not fully developed. Dissemination and Unsatisfactory The final report was not published. Discussion was confined dialogue to the main counterpart. Indirect dissemination took place, however, through the 2006 VDR. 119 Detailed Assessments: 2006 ICA Note: The tables in each section are not ratings sheets. They are tally sheets that allow the evaluator to tick off key characteristics of each AAA product. The number of ticks in each column may be summed to help inform an overall rating for each product. A. Results Comments Results indicators Did the product have… …clear results / objectives defined at Yes, it was stated in the concept paper which was shared with inception? the Government. …clearly defined indicators defined at The indicators are based on the well-known ―enterprise inception? survey‖ methodology which is applied across countries. …a clear strategy to achieve results (log- No. frame)? To what extent were the expected results and There is little institutional memory that remains to address this the results-chain explained to the client? question. Results achieved In the client country, were the findings used… …in government policy law or regulation? The Bank was involved as an advisor when the Unified Enterprise Law and the Common Investment law were drafted. The ICA was conducted at the same time, and ICA team members were also involved as advisors, and the survey findings served as one of the inputs. …in design of spending or investment No. programs? …to raise stakeholder awareness / capacity? Somewhat. There is some (but not conclusive) evidence that the interaction with the ICA team helped the government with conducting its own surveys of the business environment. …to build a coalition for change? No. …to influence the donor community? No. The Asian Development Bank (ADB) was involved at the early stages of the report, e.g., providing input to the selection of survey questions. The ICA report mentions that it was prepared in partnership with the ADB, but this seems to be overstated given limited involvement of the ADB in the finalization of the report and its aftermath. In the Bank, were the findings of the product used in …Bank strategy formulation (CAS/PRSP)? No. The 2007 CAS (as well as the 2009 CAS Progress Report) reflect some of the main messages from the ICA. …the design of Bank lending products or Not significantly. Some Policy Notes on the investment AAAs? climate were prepared for the Government while the ICA was in progress. The 2006 Vietnam Development Report has a chapter based on the findings of this ICA. There is no evidence of impact on lending. 120 Comments If there were recorded results, is there No. evidence that they are likely to be sustainable? Monitoring of impact Did the Bank or the client undertake an No. assessment of the product‘s impact? What intermediate outcomes or indicators The enterprise survey data set established a baseline from reflect the influences of the ICA? which further research could benefit. There is some (but not conclusive) evidence that the interaction with the ICA team helped the government with conducting its own surveys of the business environment. What results / outcomes are attributable to this See Discussion of Results below. ICA? Discussion of Results Results is rated Unsatisfactory. The ICA was conducted at a time when the Unified Enterprise Law and the Common Investment Law were drafted. The Bank, including ICA team members, was involved as an advisor on these topics to the Prime Minister‘s Consultative Group. The survey findings served as one of the inputs, although that link in the results chain is not very clear. The two laws represented major steps towards leveling the playing field between private and state-owned companies. The passage of the two laws, however, reflects a major campaign of the Government to win the WTO accession, which encompassed broader reforms than those envisaged in the ICA. This meant that the ICA, with its focus on leveling the playing field, and a final report that was quite delayed and not published, did not have a chance to impact the results agenda. B. Strategic relevance and ownership Comments Strategic relevance Was the delivery of the output timely and on No. The ICA‘s concept note was drafted in the second half of schedule? 2004, the final report is dated November 2006, and the government‘s comments were received late 2007. Was the issue identified as a development constraint in (note all that apply)… …the relevant CAS? Yes. The issues (SOE and finance) covered by the ICA were identified as key constraints in the Government‘s Comprehensive Poverty Reduction and Growth Strategy (CPRGS), which the Bank supported. …previous lending or nonlending work? Yes. Yes, the country team was assisting the Government on the agenda for WTO accession which was related to the ICA. …policy dialogue with clients? Yes. Policy briefs and informal advice was given regularly. …donor coordination forums? Yes. The findings of ICA served as a Chapter of 2006 VDR which was Tabled at the Consultative Group Meeting. In cases where the dialogue is difficult, did the product, did the task find a solution? 121 Comments Ownership Is the product part of an overall program of Yes. The Bank and the IFC had several lending and non- AAA to which the authorities have lending products on the IC in its strategy. The ICA formed an contributed or agreed? integrated part of this. Did the client request or commission the No. The ICA being a new product, it was proposed to the specific product? government rather than requested. This, by itself, was by no means detrimental to the ownership. Did the client cover some or all of the costs? No. Did key decision makers collaborate with, Not. But this is not surprising, given the limited personnel discuss or provide feedback on the product? with the necessary skills required for ICA work. Did a local partner (firm or public agency help to (note all that apply): …plan and design part of the work? Somewhat. Local involvement was mostly limited to the firm (Concetti) that was hired to conduct the enterprise surveys. Concetti‘s input in terms of shaping the survey (questions) seems to have been quite limited. The preparation and review of the report did involve consultations with Vietnamese researchers and practitioners who participated on a personal capacity. …analyze the results and write the report? No. …provide peer review or comments on the No. The draft ICA was not shared with Concetti for draft report? comments. …organize workshops or discussions about No. the findings? Discussion of Strategic relevance and ownership Strategic relevance and ownership is rated is rated Moderately Satisfactory. The Bank‘s initiative for the ICA and the preparation of the report took place at a time when the Vietnamese economy was rapidly opening up from a state-directed model towards a market economy, and when there was healthy momentum on the side of the government to address obstacles in the investment climate. The ICA‘s main focus of collecting and presenting perceptions of the level playing field between state-owned and private firms was a highly relevant initiative at the time. This topic, and others covered in the ICA were identified in the Bank‘s Country Assistance Strategy, lending and non-lending, and in the dialogue with the government and donors. This makes for a high relevance ex-ante, but the relevance ex-post was negatively affected by the dimension of timing. The ICA was quite delayed. The ICA‘s concept note was drafted in the second half of 2004, the final report is dated November 2006, and the government‘s comments were received late 2007. These comments were delivered at the end of 2007, at which point in time, MPI plausibly made the point that during the past two years many regulations have been improved to create a more favorable climate for doing business. Part of this tardiness can be explained by the change of TTL after the first draft of the report was prepared. As pointed out in the Discussion of Results, with the passing of the Investment and Enterprise laws, the policy agenda moved towards WTO accession, and the ICA had lost much of its relevance. Ownership could have been enhanced by greater client engagement. The case of Vietnam illustrates two dimensions of this. First, engagement and collaboration were concentrated at the beginning and tail end of the process. As this process was lengthy, it meant that the Ministry of Planning and Investment (MPI), the ICA‘s main counterpart, was left out of the drafting process and cold comment only at a late stage. Second, client 122 engagement could have involved broader collaboration. The ICA covers a variety of topics, many of which fall outside the direct responsibility of the MPI. On top of that, capacity at MPI is limited. Greater involvement from other ministries or other (semi)private organizations could have increased ownership and could have provided a more direct venue for follow-up. With the benefit of hindsight, the absence of such an approach of broader engagement contributed to the report‘s ‗sudden death‘ as soon as it was delivered to MPI. C. Technical quality Comments Quality of content Did the product… …include appropriate knowledge (i.e. make Yes. The ICA referred, for example, to the roles and scale of use of current and relevant knowledge public owned enterprises in market economies, the check and available)? balances and transparency of the Government. …cite relevant examples of practice or Yes. The ICA referred to comparators in South-east Asia and research from other countries? around the world, including results from Doing Business. …discuss the specific institutional and policy Yes, including the key constraints such as the special legal context for the issue? status of public enterprises, the lack of land market and the limited development in the credit market. …collect and analyze existing local data? Yes. The enterprise survey made use of the Census held by the General Statistical Office, and registration listings from MPI, as well as provincial tax departments‘ data. …generate new evidence? Yes, including the spread of corruption and its relatively low cost. This example is cited, for instance in the 2006-2008 CAS. …include recommendations? Yes. Recommendations are implicit (for instance, not listed as a set of recommendations or a roadmap for reform) Is the policy advice consistent with the Yes and no. The report deals very candidly and openly with preservation of social order? topics that could be perceived as sensitive. The focus on leveling the playing field is the main example, another is a chapter on dealing with regulations, officials and corruption. Vietnamese interviewees indicated that these were acceptable topics to the government. Do the recommendations include specific The report does not present a roadmap for reform. Specific actions to be taken by specific actors? actions are embedded in the main text. Was the task team staffed with the necessary The core ICA team was small and exclusively made up of expertise? FPD staff. This may have limited the scope of sector analysis. Did the product receive adequate managerial Bank staff with managerial roles at the time of the ICA could attention? not be contacted for this evaluation. That said, other interviewees did not point at a lack of managerial attention. Did the product receive a sufficient budget? Yes. TTLs do not consider the budget to be an issue, as opposed to the tensions across networks. Review of contents Was the draft peer-reviewed by appropriate Yes. experts? 123 Comments Were the peer-review comments taken into Cannot find evidence on this. account? Was the feedback from the client No. The client did not give any substantive comments. The incorporated? only remark was that the information in the report was dated. And it was true. Discussion of Technical quality Technical quality is rated Moderately Satisfactory. The Vietnam ICA is an example of a report that serves as a diagnostic tool, or a scoping exercise, rather than providing a roadmap for reform that includes a sequenced and prioritized set of policy measures. Recommendations are implicit (for instance, not listed as a set of recommendations or a roadmap for reform), and specific actions to be taken are embedded in the main text. The report reads somewhat drafty. One of the most prominent themes of the ICA is that of leveling the playing field between private and state owned firms. In addition, another is a chapter on dealing with regulations, officials and corruption. According to several interviewees, these topics were not sensitive issues to the government. That said, according to the last TTL, the reason that the report was not published was the government‘s sensitivity to some of the governance related findings. Reflecting on the ICA, the last TTL found that the ICA‘s presentation of survey findings concerning a specific sector should be accompanied by a proper sector analysis—which was not the case in the draft that he took over. Being a financial sector specialist, he re-wrote the section on land and finance. Still he found that, in terms of the selection of topics of focus, the ICA should have included an analysis of ports—as these play a vital role in the further development of the country‘s infrastructure to support the investment climate. D. Dissemination and sustained dialogue Comments Dissemination Was the product… …made available in the local language? Yes. But probably not distributed beyond MPI. …made available on a website? No. The ICA was never published, but the enterprise survey results are publicly available. …discussed with senior policy-makers? Yes. The ICA‘s first TTL presented and discussed the draft report with MPI officials. …presented at a workshop, conference or on No. The contents of the report were no longer relevant when it the web? was completed. No dissemination was done, although some of the findings were presented in a chapter of the 2006 VDR. …covered in the media? NO. Sustained dialogue Did the product help sustain engagement through… 124 Comments …policy dialogue? NO. …discussions/contact with stakeholders? NO. …lending and non-lending products? NO. Some Policy Notes on the investment climate were prepared while the ICA was in progress. There is no evidence of any impact on lending. …other means (please discuss below)? Discussion of Dissemination and sustained dialogue Dissemination and sustained dialogue is rated Unsatisfactory. As mentioned above, the report was never published. Several reasons have emerged from interviews. Each of these reasons does not necessarily rule the others out. The most plausible reason would be that after absorption into the VDR and departure of the original TTL, there was no real momentum or need to publish the ICA as a separate document. In a way, this can be verified as the VDR indeed includes survey findings. Another reason brought forward in interviews is MPI‘s objection to outdated data used in the draft ICA. This message was transmitted to the Bank in MPI‘s written comments to the draft ICA. These comments were delivered at the end of 2007, at which point in time, MPI plausibly made the point that during the past two years many regulations have been improved to create a more favorable climate for doing business. In addition, the last TTL argued that the ICA was never published because of the sensitive nature of the findings on dealing with regulations, officials, and corruption. The ICA‘s first TTL presented and discussed the draft report with MPI officials, and the ICA‘s second TTL delivered the final draft to MPI. Beyond that, no dissemination or dialogue took place. In fact, there is no evidence that MPI shared the ICA report with any organization outside of the ministry. That said, the 2006 VDR which contained part of the ICA findings was launched at the annual Consultative Group Meeting and attracted considerable attention. In this context, it is worth pointing to two initiatives of flagship analytical reports conducted outside the Bank, which attracted considerable attention. These are: 1. The Vietnam Provincial Competitiveness Index, and 2. A recent report on Vietnam‘s investment climate to which Michael Porter contributed. 125 ANNEX 6: LIST OF PEOPLE MET Bangladesh: SK. MD. Abdul Ahad Director, Prime Minister‘s Office, Bangladesh Md. Arifu Haque Deputy Director, Board of Investment, Prime Ministers, Office. Mahbubur Rahman Member Finance, Bangladesh Export Processing Zones Authority (BEPZA), Bangladesh Abu Reza Khan Member (Engineering), BEPZA (Former BOI member), Bangladesh Professor Mustafizur Rahman Executive Director, Center for Policy Dialogue (CPD), Bangladesh Debapriya Bhattacharya Distinguished Fellow, Center for Policy Dialogue (CPD), Bangladesh Mr. Aslam Alam Secretary In-Charge, Ministry of Food & Disaster Management, Bangladesh M Humayun Kabir Former Ambassador to USA, Former Secretary to the Government of Bangladesh Farooq Sobhan President, Bangladesh Enterprise institute, Bangladesh Najmul Hossain Managing Director, Data International Limited, Bangladesh. Azizur Rahman Director, Data International Limited, Bangladesh. Syed Abu Naser Bukhtear Ahmed Managing Director, Trade Hub (Bangladesh) Limited, Bangladesh Rumee Ali Managing Director, BRAC Bank, Bangladesh Mahbubur Rahman, President, International Chamber of Commerce Bangladesh Mrs. Laila Kabir, Former BBBF member, Bangladesh Dr. Ali Akbar Khan Bangladesh Egypt: Ms. Mona Zobaa Advisor, Ministry of Investment, Egypt Dr. Manal Metwaly Director, Center for Economic and Financial Research Studies (CEFERS), Cairo University, Egypt Dr. Ramadan Hamed Social Research Center of the American University in Cairo, Egypt. Mr. Sameh Sewillam Head of Research, the American Chamber of Commerce Ms. Nevine El Shafie Vice Chairman, General Authority for Investment and Free Zones Guatemala: Guillermo Monroy World Business Center for Sustainable development (CENTRARSE) Julio Héctor Estrada General Manager – Palo Blanco Investments (ex-director Pronacom), Emanuel Seidner Ex-Director, Comisión de Competitividad 126 Jorge Lavarreda Vice President of the Board, Center of National Economics Research, Francisco Marroquin University, Guatemala Rodolfo Batres PDC – Grupo Capital Ligia Chinchilla Quiej de los Bosques Juan José Cabrera Federico Pola Gerente, Cámara De Comercio Abel Cruz Mineco (Ministry of Economy) Jason Steele Gerente de Proyctos, GSI Karina Posadas Gerente Eficiencia Adminsitrativa Del estado, PRONACOM Roberto Mancilla Asesor, PRONACOM Gabriela Garcia NGO Guatemala Proyecto de Vida (former staff of Pronacom 2006-2010) Marcio Cuevas Mexico y Centroamerica, Walmart Mario Marroquín Director Ejecutivo Guatemala, Montana Exploradora (former Bank staff and former director of Invest in Guatemala) Mario García Lara Director Ejectutivo, Fundación 2020 Lic. Pablo Schneider Director, Central de Inciativas De Desarrollo – CIDES, FUNDESA Kenya: Hon. Ndiritu Muriithi Assistant Minister, Ministry of Industrialization, Kenya Ms. Betty Maina Chief Executive Officer, Kenya Association of Manufacturers (KAM), Kenya Mr. Matanda Wabuyele Chief Executive Officer, Export Promotion Council, Kenya Ms. Carole Kariuki, Chief Executive Officer, Kenya Private Sector Alliance, Kenya Ms. Susan Kikwai Managing Director, Kenya Investment Authority, Mr. Wycliffe Wasike General Manager, Kenya Investment Authority Ms. Sheila Yieke General Manager, Legal and Corporate Affairs, Kenya Investment Authority Dr. Moses Ikiara Executive Director of the Kenya Institute for Public Policy Research and Analysis (KIPPRA), Kenya Hon. Mithika Linturi Member of Parliament and Member of Public Investment Committee, Kenya Hon. Yusuf Chanzu Member of Parliament and Member of Public Investment Committee, Kenya Mr. Tim Lamont Economist, DFID, Kenya Mr. Mageto Ministry of Trade, Private Sector Working Group, Kenya Mr. Joseph Kinyua Permanent Secretary Ministry of Finance-Treasury, Kenya Vietnam: Mr. Bui Thu Thuy Director, Policy and General Issues Division, Enterprises Development Agency. Ministry of Planning and Investment. Yoshifumi Tsujio Senior Investment Advisor, Foreign Investment Agency, Ministry of Planning and investment. 127 Mr. Dang Duc Dzung President, Hung Viet consult, JC. And Vice Chairman of Young Business Association, Vietnam Mr. Hayashida Takayuki, Senior Project Formulation Advisor, Japan International Cooperation Agency (JICA), Vietnam Mr. Murooka Naomichi Representative, Japan International Cooperation Agency (JICA), Vietnam Mr. Hank Tomlinson President, Chevron Vietnam Limited Mr. Dau Anh Tuan Deputy Director, Legal Department, Vietnam Chamber of Commerce, Vietnam Mr. Ayumi Konishi Country Director for Vietnam, Asian Development Bank, Vietnam Tran Thanh Son Founder and CEO, Tamtay Vietnam Mr. Han Manh Tien Director, Consulting and Research Company for Technology Transfer and Investment (CONCETTTI), Vietnam Dr. Ho Sy Hung Director General, Enterprise Development Agency, MPI, Vietnam Mr. Pham Dinh Thuy Director of Industrial Department, General Office of Statistics, Vietnam Ms. Dam Thuy Nga Deputy Director of Executive Office in charge of Internal Affairs, State Capital Investment Corporation (SCIC), Vietnam Mr. Nguyen Dinh Cung Vice President, Central Institute of Economic Management, Vietnam Ms. Duong Thu Huong General Secretary, Member of Board of Directors, Secretary- General, Vietnam Bankers Association, Vietnam Mm. Pham Chi Lan Member of former Prime Minister Research Group, Vietnam Mr. Bach Van Mung Director of Vietnam Competition Authority, Ministry of Trade and Industry, Vietnam Mr. Preben Hjortlund Managing Director, Hanoi International Technology Center (HITC) Mr. Dang Duc Dzung Vice Chairman of Young Business Association Bach Van Mung Director General, Vietnam Competition Authority, Ministry of Industry and Trade Trinh Anh Tuan Head, International Cooperation Board, Vietnam Competition Authority, Ministry of Industry and Trade Nguyen The Ngan Analyst, Strategy Department, State Capital Investment Corporation (SCIC). Nguyen Thi Kim Anh Analyst, Strategic Department, State Capital Investment Corporation (SCIC). Nghiem Diep Thoung Assistant to Secretary General, Vietnam Bankers Association. Mr. Jay Kryk Economic Growth Office Director, USAID- Vietnam. 128 World Bank Staff: A. K. M. Abdullah Financial Sector Specialist, World Bank, Bangladesh G. M. Alam Khurshid Senior Private Sector Development Specialist, World Bank, Bangladesh Syed Estem Adul Islam Monitoring and Evaluation Officer, Advisory Services in South Asia, IFC, Bangladesh. Dr. Sahar Nasr Lead Financial Economist, World Bank, Egypt Andrew Stone Lead Private Sector Development Specialist, World Bank Egypt Fernando Paredes Operations Officer, World Bank, Guatemala Jane Kiringai Senior Economist, World Bank, Kenya Tracey Lane Senior Economist, World Bank, Kenya Wolfgang Fengler Lead Economist, World Bank, Kenya Johnnes Zutt Country Director, World Bank, Kenya Nguyet Anh Senior Energy Specialist, World Bank, Vietnam Victoria Kwakwa Vietnam Country Director, World Bank, Vietnam Sameer Goyal Senior Financial Sector Specialist, World Bank, Vietnam Paul Vallely Transport Sector Coordinator, World Bank, Vietnam Nguyet Anh Senior Energy Specialist, World Bank, Vietnam Geeta Batra Chief Evaluation Officer, Independent Evaluation Grp-IFC Michael Goldberg Senior Private Sector Development Specialist, World Bank