Improving Efficiency in Public Procurement in Georgia Findings from Data Analysis of Public Procurement Transactions in Georgia 2013-2016 Joel Turkewitz Sandro Nozadze Stephen Davenport Fredrik Sjoberg Jonathan Mellon Mark Brough Hunt La Cascia Mediha Agar June 2018 Improving Efficiency in Public Procurement in Georgia Improving Efficiency in Public Procurement in Georgia Findings from Data Analysis of Public Procurement Transactions in Georgia 2013-2016 Joel Turkewitz This project was funded with UK aid Sandro Nozadze from the UK Government Stephen Davenport Fredrik Sjoberg Jonathan Mellon Mark Brough Hunt La Cascia Mediha Agar 2 Photo by Kimon Maritz on Unsplash June 2018 I Introduction Improving Efficiency in Public Procurement in Georgia O ver the course of the last decade, Georgia has made remarkable progress in estab- lishing a strong public procurement system. New legislation in 2009 provided a well-structured and unified regulatory basis for public procurement. The adoption of an e-procurement system in 2010 introduced modern technology to the procurement system and served to improve and standardize processes for undertaking transactions and organizing tenders. The creation of a new independent State Procurement Agency in 2001 strengthened oversight of public procurement and the overall management of the public procurement system.1 Each of these actions has contributed to the creation of a public procurement system that has received international recognition for its excellence by or- ganizations such as the Asian Development Bank, the European Bank for Reconstruction and Development, and the United Nations. The structural and procedural improvements in the public procurement system since 2009 are clear and evident. The extent to which those improvements have translated into better performance has been the subject of limited external analysis and assessment, but has received increasing attention as Georgia strives to ensure that it is making the best use of public resources. Improvements in efficiency or effectiveness of processes can have substantial consequences for the national budget as well as the national economy, given that procurement represents over 30% of public spending and slightly less than 10% of GDP in Georgia.2 The current study was designed to create a better understanding of the performance of the Georgia public procurement system from 2013 to 2016. Guided by specific concerns identified by officials from the Government of Georgia, the analysis examined a limited number of performance variables relating to the efficiency of the procurement process, and the value for money obtained through the procurement process.3 Overall, the study demonstrated that the public procurement system functions at a high level of effectiveness, and is able to fulfill its core functions. At the same time, the study findings and recommen- dations identity a number of issues which present opportunities for improving outcomes in the short to medium term. The study approach, of using transaction level data to examine intermediate and final outcomes of the procurement system, may also serve to catalyze similar work in the future, and enable a rigorous and well-informed policy dialogue on how well the Georgia public procurement system meets the needs of the country and its people. 3 This note provides a brief overview of the size and composition of public procurement. It then examines performance in relation to two key outcome variables – the success rate of competitive tenders, and the level of competition in open procedures. The nature of Improving Efficiency in Public Procurement in Georgia the performance issues in these two areas are explored, and specific recommendations are developed for improving performance in the short to medium term. A different per- spective on performance is then provided through examining the degree to which small and medium firms participate in procurement tenders and are awarded contracts. A final section provides a limited number of recommendations on steps to establish a continuous process of data analysis and performance evaluation. 4 II Methodology Improving Efficiency in Public Procurement in Georgia T his study presents the findings of analysis of the database of procurement trans- actions for 2013 through 2016 obtained from the State Procurement Agency. The database captured information on 878,900 state contracts that went through ei- ther simplified processes or were the subject of e-tenders.4 Procurement transactions not covered by the Public Procurement Law, and therefore not recorded in the SPA database were not reviewed.5 The value of the contracts in the data base amounts to 12,323,089,060 GEL. The transaction database was supplemented with additional information on firm size that was obtained from the Ministry of Finance. Based upon data limitations, the evaluation of performance focused exclusively on indica- tors derived at the start of the procurement process – what the state elected to purchase, the extent of competition for state contracts, and the nature of the firms that competed and won state contracts. Performance was assessed largely on the basis of data relating to frequency of failed tenders and the contract price. Analysis of data on contract imple- mentation and asset management would enable a more robust assessment of the value for money obtained from procurement, and provide a deeper performance profile. Future studies may seek to expand the scope of inquiry in these directions providing that accurate and comprehensive data on these areas exists and can be made accessible. Preliminary study findings were presented to the government officials at a one-day work- shop held in February 2017, and were discussed in greater detail in face-to-face interviews. 5 III Overview of the Public Procurement System of Georgia Improving Efficiency in Public Procurement in Georgia Size, Composition, and Method of Procurement D ata on the composition and value of procurement transactions from 2013 to 2016 reveals a procurement system that is characterized by a number of consistent patterns, as well as areas of relatively volatile change. Overall, an average of almost one third of all public spending (31.7%) took place via public procurement across the four year period, with little year-to-year variation.6 Procurement spending represented 9.9% of GDP across the same period, with also a high degree of consistency across all years.7 These figures demonstrate the core role of the Government of Georgia as a purchaser of goods, works, and services, and the degree to which public sector performance, and overall economic performance in Georgia, is linked to the per- formance of the procurement system. The size of the procurement market in Georgia also implies that even slight performance improvements can generate substantial reductions in expenditures and generate important economic stimulus for private sector development. Ministries consistently procured the largest proportion of value of procurement and edu- cational authorities consistently procured the least. While State-Owned Enterprises (SOEs) and Municipalities began and ended the period relatively unchanged in regard to the pro- portioned value of their procurement purchasing (between 20 and 25% of value), they each experienced a high degree of volatility in the interim years. The rise in value of municipal procurement as a proportion of spending in 2014 and 2015 was mirrored by a corresponding decline in the proportion of the value of procurement associated with SOEs in those years. Those trends were reversed in 2016 with a sharp decline in municipal procurement and an equally sharp increase in SOE procurement. The trend lines suggest that procurement spending by municipalities either substitutes or competes with procurement spending by SOEs, in a process that presumably is reflected in changes in budget allocations and public sector priorities. See Figure 1. 6 Figure 1. Spending by Procurement Entity Type 2013 2014 2015 2016 Improving Efficiency in Public Procurement in Georgia 40% Ministry Procurement Spending 30% SOE Municipality 20% Other 10% Educational Facilities 0 A similar degree of consistency exists in regard to the composition of procurement. Across all four years, a majority of procurement transactions were for goods. Goods transactions accounted for between 55 and 60% of the volume of transactions during the entire period. A different picture emerges in regard to the value of procurement associated with goods, works, and services, with procurement for works consistently composing the largest part of overall procurement spending. Spending on works contracts increased relative to spending on other types of procurement substantially over the period, with a 10% increase recorded from 2014 to 2016. The proportion of spending associated with goods and services starts and ends at relatively the same position, but again follows inverse trends in the interim years. Overall, procurement in Georgia often takes the form of a large number of relatively small contracts for goods, and a relatively small number of larger contracts for works. This procurement composition pattern mirrors that found in many other countries. See Figure 2. 7 Figure 2. Number and value of contracts in goods, works and services, by year Contracts in goods, works and services by Year Improving Efficiency in Public Procurement in Georgia 60% 50% 40% 30% 20% 10% 0 2013 2014 2015 2016 Procurement Value in goods, works and services by Year 50% 45% 40% 35% 30% 25% 20% 2013 2014 2015 2016 Services Works Goods An alternative way of characterizing the public procurement system focuses on the method used to execute procurement transactions. Throughout the period, most contracts con- tinued to be assigned through simplified methods, with the proportion of simplified pro- curement declining during the 2013-2016 period but still accounting for over 88% of all contracts in 2016. See Figure 3. At the same time, the e-tendering method was increasingly used to execute higher value contracts. The proportion of value associated with e-tendering reached over 70% in 2016. The jump in procurement value going through e-tenders be- tween 2015 and 2016 suggests that changes introduced in 2015 to restrict the use of special procedures in higher-value contracts has achieved its intended effect. Overall, Georgia is increasingly evolving towards a two track system, with smaller contracts going through simplified procedures and larger contracts being e-tendered. 8 Figure 3. Number and value of contracts by method of procurement, by year Improving Efficiency in Public Procurement in Georgia By number of contracts 100% 90.4% 90.6% Percentage of contracts 75% 88.3% 88.3% 50% 25% 9.6% 9.4% 11.7% 11.7% 0% 2013 2014 2015 2016 By value 100% 70.3% Percentage of contracts 75% 64.8% 60.3% 51.3% 50% 48.7% 39.7% 25% 35.2% 29.7% 0% 2013 2014 2015 2016 E-tender Simplified 9 IV Overview of the Performance of the Procurement System of Georgia Improving Efficiency in Public Procurement in Georgia Level of Competition and Success Rate of Tenders A n overview of the performance of the procurement system can also be derived from an analysis of the transaction-level data.8 Guided by a focus on identifying issues concerning system outcomes, high-level performance variables centered on two issues: the proportion of procurement tenders that conclude with the award of a contract, and the level of competition that procurement tenders attract. Awarding contracts through competitive tenders is a relatively time consuming and cost- ly undertaking. Costs are borne by both the public sector and those private sector firms that elect to submit bids. At the same time, competitive tenders can generate substantial benefits in enabling the government to capture cost savings generated by competition as well as spurring on innovation in the private sector. Tenders that do not result in signed contracts can end up being quite expensive, given the wasted expenses associated with the failed procedure as well as the costs to the government of being unable to purchase something that it has determined that it needs.9 Open tenders are also costly when they do not attract competition since the government pays for the cost of organizing the tender but fails to derive the expected benefits from competition. A review by Estache Antonio and Atsushi Iimi of 211 procurement auctions for large contracts in the road, power, and water sectors from 1997 to 2007 in 29 developing countries found that the value of competition increased with the number of bidders and that the full benefits of competition for large contracts were achieved in auctions that attracted 6 or more bidders. 10 This finding on the optimal level of competition for large contracts confirmed the results of other empirical studies that are presented in the Antonio Estache and Atsushi Limi paper. The state does not capture the full advantage of open tenders in infrastructure in those situations where fewer than 6 bids are received. Data on the failure rate of procurement opportunities that went through e-tendering are concerning. Over the four-year period, an average of 29.1% of e-tenders did not result in a signed contract. This average has remained relatively constant over the time period, with some decline in 2016. 10 Figure 4. Failure rate by contract Improving Efficiency in Public Procurement in Georgia 29.1% Fail Competition in e-tendered contracts for the period 2013 to 2016 has also not achieved a desired level. The average number of bids for open tenders has not surpassed 3 in any year. Tenders for works contracts attract the largest number of bids, but competition for goods and works contracts has been minimal. See Figure 5. Overall, the level of competition for procurement contracts over the last four years has been substantially below what is needed in order for the Government to derive maximum benefit from private sector competition. Figure 5. Level of competition for contracts in goods, works and services Mean Bidders 0 1 2 3 4 5 6 Goods Services Works Mean Bidders in each procurement category Tender > 5,000 GEL only The data on failed e-tenders and competition levels has identified two areas where current outcomes could be improved.11 Increasing the proportion of tenders that result in a signed contract, and increasing the level of competition for state contracts would save money and improve the value for money obtained from the system. We turn now to developing rec- ommendations on how to address each of these problems based upon additional analysis of the transaction-level data. 11 V Improving Outcomes Improving Efficiency in Public Procurement in Georgia Raising the Success Rate of Contracting T he performance review highlighted continuing issues with attempts to execute procurement transactions that fail to result in signed contracts. Analysis of failed tenders reveals that they are strongly associated with the size of the contract op- portunity – tenders between 5,000 GEL and 200,000 GEL have a failure rate of 29.8% while tenders above 200,000 GEL fail at a rate of 18.8%. Not surprisingly, smaller contracts also attract less competition, averaging just over 2 bids per tender, as opposed to just over 3 bids per tender for larger contracts. No substantial regional variation was detected in failure rates, and the proportion of failed contracts suggests that failure is not strongly associated with the procurement of one or more particular good or service.12 See Figure 6. Figure 6. Failure rate and number of bidders for small and large contracts. Size of bubble shows total value of contracts. Percentage 30 of Failure 25 20 15 2.0 2.5 3.0 3.5 Mean number of bidders The poorer performance of tenders for smaller contracts serves to bring into focus the degree to which small contracts dominate current procurement transactions. In goods procurement, over 90% of transactions were for contracts of 5,000 GEL or below. Service contracts have a similar distribution with few contracts valued above 200,000 GEL. There is a population of works contracts that are valued above 200,000 GEL but that population accounts for under 10% of the transactions relating to the construction and maintenance 12 of capital work. Figure 7. Number of contracts by size, for goods, works and services 0 50% 100% Improving Efficiency in Public Procurement in Georgia 10% 20% 30% 40% 60% 70% 80% 90% Goods Services Works Contract Size 0-5,000 5,000-200,000 200,000+ Despite their overwhelming numbers, small contracts account for only a small portion of the value of procurement spending. The 90% of goods contracts that are 5,000 GEL or smaller account for less than 10% of the value of spending on goods. A similar distribution is found in service contracts. Just under 80% of the value of works procurement is associ- ated with the slightly less than 10% of contracts that are above 200,000 GEL. See Figure 8. Figure 8. Value of contracts by size, for goods, works and services 0 10% 20% 30% 40% 50% 100% Goods Services Works Contract Size 0-5,000 5,000-200,000 200,000+ Overall, it appears that the bulk of the time and effort involved in public procurement is devoted to processing very small transactions. This drain on resources and efficiency is compounded by the significantly higher rate of failure for small contracts that are com- petitively tendered compared to success rate for larger contracts that are competitively tendered. Actions that would reduce the number of small contracts could improve overall allocation of resources within the public procurement system, as well as reduce the costs associated with failed tenders. An indication of the possible savings that could be obtained through consolidation is provided by the teams calculation that reducing the number of 13 small contracts by finding ways to combine contracts valued at 200,000 GEL or more would generate direct benefits of more than 1.7 million GEL, based on increased cost sav- ings associated with larger contracts and a reduction in the volume of contracts by over Improving Efficiency in Public Procurement in Georgia 27,000.13 The proportion of failed contracts would also decline given the 11% greater fre- quency of failure of tenders for contracts below 200,000 GEL. Experience in other countries suggest a variety of approaches to reducing the number of small contracts. Many countries have introduced framework contracts to restructure the manner in which they may purchase goods, especially standardized items, such as paper, or other supplies, that have traditionally been bought in high volume but with low value. Framework contracts are structured in a variety of ways, but they share a common ob- jective of eliminating the need to form a contract for every individual purchase.14 Instead, contracting authorities can make use of one or more underlying contracts that have been previously negotiated in order to simply obtain the required goods. Framework contracts, often negotiated and signed by a central authority, serve to enable the government to make full use of its bargaining power, capture large economies of scale, and increase the trans- parency of operations. Given these advantages, it is not surprising that many countries have elected to make extensive use of such arrangements. ChileCompra, the contracting authority that handles framework contracts, for example, has contracts for over 160,000 products. The Crown Commercial Service, the UK contracting agency responsible for sign- ing and managing framework contracts at the municipal level, maintains relationships with over 2,600 suppliers. Framework agreements can be particularly influential when they are paired with the adoption of a catalogue, which eases the ability of government offices to identify and purchase goods. Application of modern information technology has led to the creation of eCatalogues in many countries, which has made purchasing of goods a simple, quick, and efficient process. The introduction of framework contracts, with or without eCatalogues has often generated substantial savings on the unit cost of goods. At the same time, framework contracts have reduced the volume of transactions and helped to focus resources on higher value and more complex procurement. ChileCompra estimates that their savings average 7-10%. The U.K. estimates that they have achieved 27.4% savings, while the Office of Government Procurement in Ireland calculates that they have achieved cost reductions of up to 30% on the purchase of common office supplies, including computers. Framework contracts are not new to Georgia. They have existed for some time and are used on a limited basis.15 There appears to be extensive space to expand their application. Additional study by the SPA of their experiences with framework contracts and the factors 14 that have constrained their utilization might provide the information needed in order to make better use of this contract form. Georgia also has some experience with creating catalogues, but the current approach uses catalogues for informational purposes.16 Improving Efficiency in Public Procurement in Georgia Countries have taken a slightly different approach to improving the efficiency of con - tracting for works. Alternative approaches to increasing the size of public works contracts include increasing the value of contracts by moving towards multi-year engagements for repeated and predictable activities (such as road or irrigation maintenance) or broaden- ing the geographic range of work to include activities in multiple villages. A substantial amount of experience has been gained in other countries concerning how to aggregate works contracts in ways that reduce costs and add value. The common feature of each of these innovations is the movement away from treating each state purchase as a unique event. Instead, planning allows the state to develop an estimate of its purchasing needs and to organize its purchasing in a manner that allows it to get best value for money. Governments need to make choices regarding balancing overall cost savings with the desire to support small and medium enterprises, and a number of other considerations, but those choices can be made most effectively when they are determined by policy and not purchase-by-purchase. The specific actions that would best improve performance relating to contract size can best be determined by the Government of Georgia. A useful plan is likely to include greater use of e-catalogues, framework agreements and modification of the approach to works contracts. Regardless of how it is accomplished, re- ducing the number of small contracts would serve to improve the success rate of contracting and enhance the effective allocation of resources across the public procurement system. Recommendation: The Government of Georgia should take concrete steps to improve the efficiency of public procurement, with a focus on rationalizing the procurement of goods and works. The Government may wish to set a specific target (e.g. reducing the number of small contracts by 30% over the next 3 years) in order to highlight its commitment to improving outcomes and in order to enhance monitoring. The SPA should take on responsi- bility for monitoring change and identifying and working with contracting authorities that disproportionately rely on small contracts for their purchasing requirements. Extending the use of e-catalogues and framework agreements for commonly procured goods can be a central part of this effort, and can be structured in ways that reduce transaction costs while still maintaining the ability of small and medium sized enterprises to bid and per- form contracts. Modification of the approach used for small works contracts might also be usefully included in this stream of work. The SPA should track the extent to which changes in contract size affect the failure rate of planned procurements to ensure that the reforms are having their intended effect. 15 VI Improving Outcomes Improving Efficiency in Public Procurement in Georgia Increasing Competition The second issue identified in the performance review was the relatively low level of competition for state tenders. Transaction-level data on public procurement for Georgia demonstrates substantial additional cost savings associated with each additional bidder, with savings rising particularly steeply as the number of bidders increases from 1 to 6. See Figure 9. The Government of Georgia could reap substantial benefits by raising the number of bidders in open tenders beyond its current average of slightly less than 3. Figure 9. Average percentage of value saved by number of bidders, for works contracts Value Saved 50% 40% 30% 20% 10% 5 10 15 Number of Bidders The potential financial gains to competition rise with the value of the procurement tender. As contract value climbs, the value that can be saved when more firms bid also rises. As discussed above, higher value contracts in Georgia are primarily associated with works contracts. Data analysis reveals that high-value works contracts are associated with a very limited number of organizations – the top 10 entities on the basis of the value of their spending on public works account for 25% of the entire procurement budget, and over 50% of all public procurement spending on capital investment. See Figure 10. A focus on 16 improving competition within these 10 organizations, therefore, could potentially influ- ence transactions that represent 25% of the value of all procurement and the majority of capital spending. Improving Efficiency in Public Procurement in Georgia Figure 10. Top ten organisations that procure the largest value of works contracts, by method of procurement e-tender simplified Ministry of Regional Development and Infrastructure of Georgia, Roads Department of Georgia 91.7% 8.3% 1,017,600,465 GEL Tbilisi City Hall 76.7% 23.3% 401,651,985 GEL LLC Gardabani TPP 12.6% 87.4% 277,621,566 GEL Educational and Scientific infrastructure Dev Agency 75.6% 24.4% 198,388,918 GEL Georgia Municipal Development Fund 82.5% 17.5% 167,636,086 GEL LTD “Georgian United Land Reclamation Systems Company” 93.3% 6.7% 156,903,396 GEL Batumi City Hall 94.5%5.5% 149,491,656 GEL LTD “SportMshenService“ 0.5% 99.5% 132,848,122 GEL The Ministry of Internally Displaced Persons from the Occupied Territories , Accommodation and Refugees of Georgia 91.1% 8.9% 105,112,739 GEL LTD “Georgia Gas Transportation Company“ 93.6% 6.4% 81,783,897 GEL A review of performance within these top 10 organizations indicates wide performance variations. See Figure 11. Differences exist in the degree to which these organizations rely on open tenders, the competiveness of their competitive tenders, and the cost savings on procurement that they have been able to realize. For example, the proportion of tenders with more than one bidder ranges from approximately 50% to above 90%, while average savings ranges from a low of under 10% to a high of almost 30%. 17 Figure 11. Performance of top 10 works procuring organisations, by percentage of failed tenders and competitive contracts Tibilisi Educational and Scientific Infrastructure 20 Improving Efficiency in Public Procurement in Georgia Ministry of IDP Roads Department Municipal Development Fund Percentage of failed tenders 15 Gas Transportation Company Batumi 10 United Land Reclamation Systems Company 5 LLC Gardabani TPP 0 40 60 80 100 Percentage of e-tenders with more then one bidder It is essential to remember that performance variation occurs across organizations that do procure a substantial volume of capital works, even though all of the organizations operate under the same legislative rules and policies. While differences exist in the precise type of capital work being procured, organizational practices, skills, and behaviors are likely to be important factors in determining exactly where an organization’s procurement outcomes rank in comparison to the others in the group. Improving outcomes will be achieved when the organizations that have the weakest outcomes can bring their routines and capabilities closer to those of the best performing organizations. Georgia could save approximately 370,000,000 GEL if all of the top 10 organizations achieved the same savings as the cur- rent best performer.17 Alternatively, attracting an average of 4 bidders for works contracts tendered by the top procuring entities would result in a 1.8% saving (37,352,089 GEL). Attracting an average of 6 bidders would raise savings to 6% (125,474,790 GEL). Undoubtedly, the single most valuable improvement would be to strengthen outcomes in the Roads Department of the Ministry of Regional Development and Infrastructure of Georgia. The Roads Department is the largest procurer of capital works in the country. It procures approximately the same value as the combined total of spending on works of the third through tenth largest procurers. Unfortunately, it is currently one of the lower performers, attracting more than one bidder in slightly less than 60% of its tenders, and achieving savings of less than 10%. Unlike large procurement in other organizations, the level of competition does not consistently increase with the size of the contract. See Figure 12. 18 Figure 12. Number of bidders against tender value, for Roads Department Number of Bidders Improving Efficiency in Public Procurement in Georgia 4 3 2 1 1,000,000 10,000,000 Tender Value (GEL) The World Bank is working with the Government of Georgia on improving the overall man- agement of public investment. Much of that work has focused on improving the planning of projects and creating a more consistent and rigorous process for determining which projects merit public resources. Improving investment planning may be an important step in increasing competition for works contracts overall. Data on participation rates in procurement tenders show that slightly over 20% of large firms bid for state contracts, and that this percentage is declining over time. See Figure 16 for information on firm participation rates. This low level of participation suggests that the problem is not with the size of the private sector but the interest of large private sector firms in pursuing contract opportunities. The World Bank has assisted countries such as India and Indonesia to undertake market studies in their roads sectors when they were facing similar competition issues. These studies identified a range of issues from concerns about poor design and specifications, to the tendency for contracts to require private firms to take on all the risk relating to implementation problems, to long delays in having work approved and receiving payment. Increasing competition for contracts was understood to require changing a number of practices relating to bidding, contracting, and contract management. 19 A final possible cause of low levels of competition may be the existence of collusive agree- ments among firms. Agreements among firms to “share” markets, with or without the knowledge and involvement of government officials, have been found in many countries. Improving Efficiency in Public Procurement in Georgia Collusion is often particularly strong in capital works, due to the limited number of market players and the amount of money involved. While Georgia has been exceptionally success- ful in its anti-corruption work, there may be scope to employ techniques such as forensic audits and analysis of bidding behavior, to evaluate the role of collusion in shaping com- petition for public works contracts.18 Improving competition for public works contracts in general, and roads contracts in par- ticular, is likely to require the combined efforts of several ministries in Georgia. The World Bank and the Ministry of Finance are already engaged on improving public investment management. The Bank and the Department of Roads are also working together on im- proving the efficiency and effectiveness of the management of the road network. A focused effort to increase competition in procurement, supported by the SPA, would complement the existing initiatives and may contribute greatly to improving outcomes. RECOMMENDATION The Government of Georgia should undertake a program to improve the level of compe- tition for large works contracts, with an initial focus on improving competition in pro- curements associated with the top 10 procuring entities. The Government of Georgia may consider setting a target of achieving an average of 4 bidders per tender within 3 years. Georgia should specifically prioritize improving competition in the Roads Department. Responsibility for acting on this recommendation should be with the Administration of the Government of Georgia, the Roads Department of the Ministry of Regional Development and Infrastructure of Georgia, and the Municipal Development Fund of Georgia, working with the support of the SPA. 20 VII Improving Outcomes Improving Efficiency in Public Procurement in Georgia Improving Procurement Performance at the Municipal Level P rioritizing efforts to reduce the number of small contracts and improve competi- tion in large works contracts has been justified by the potential returns in regard to improved efficiency and cost savings. A third priority, improving procurement performance at the municipal level is partially driven by the importance of municipal pro- curement for local economies and for service provision. While procurement in ministries and state owned enterprises has the greatest impact on the budget, procurement at the municipal level has the greatest impact on the lives of people. Effective local procurement can be a catalyst for private sector development, and can enable local governments to provide public services in a timely and accessible fashion. Poor procurement can do just the opposite. Until 2016, the proportion of procurement undertaken by municipalities grew steadily. It is not clear whether the sharp decline in 2016 reflect a change in course or a single year anomaly. Municipalities can be divided into two clusters based upon their procurement performance – a cluster that performs better and a cluster that performs worse. Better per- forming municipalities have fewer failed tenders and achieve higher levels of cost savings. See Figure 13. They also feature a higher degree of competition in procurement – using e-tendering for a higher proportion of their procurement purchasing and attracting more bidders to their tenders. See Figure 14. 21 Figure 13. Under-performing and high-performing municipalities, by failed tendering and average savings Kazbegi Khobi Improving Efficiency in Public Procurement in Georgia Percentage point difference in failed tendering Samtredia 0.10 Tigvi Kurta 0.05 Mestia Khashuri Tianeti Shuakhevi Martvili 0 Bagdati Chiatura Tblisi -0.05 Dmanisi Sagaredjo Rustavi Ninotsminda Sachkhere Chkhorotsku Terdjoia Chokhatauri Akhalkalaki -0.10 -0.03 0 0.03 Percentage point difference in average savings Figure 14. Characteristics of high-performing and under-performing municipalities: procurement method and mean number of bidders Procurement Spending Mean number of bidders on e-tenders 0 20% 40% 60% 80% 0 0.5 1.0 1.5 2.0 2.5 E-tender E-tender Simplified System Municipality Performance Under Performing High Performing It may be that the difference in performance is tied to the wealth of the municipality and the depth of the local private sector market. A thorough investigation of the factors that distinguish better performing cities from those that perform worse is outside the scope of this study. Performance varied widely among municipalities within each region, so there was no clear regional explanatory factor. See Figure 15. 22 Figure 15. Difference in average savings on procurement, by region Difference in Average Savings -0.05% 0% +0.05% Improving Efficiency in Public Procurement in Georgia Region Ajara Guria Imereli Kakheli Kvemo Karli Mtskheta Mtianeli Racha-Lechkhumi and Kvemo Svaneli Samegrelo Zemo Svaneli Samtskhe-Javakheli Shida Kartli Tbilisi Similar to the situation among the large procuring entities, variations in performance among municipalities cannot be explained by differences in regulations or legislation. Instead, the factors that generate large variations relate to the manner in which those rules are implemented and the capabilities of local government and the private sector. Municipalities that rank in the bottom third in regard to savings could save an additional 5.4% if they could raise their performance to that of the better performing cities. The additional value that could be obtained to local firms cannot be calculated but is likely to also be significant. Georgia is not unique in experiencing large variation in municipal procurement per- formance. A number of countries have tried innovative approaches to achieving better outcomes. The Philippines undertook an extensive program of procurement training to increase the capacity of procurement officials across the country to fulfil their functions. The UK has worked to establish links between communities and municipalities, in order to achieve better outcomes through the combination of skills and expertise in different locations. Bangladesh, with the help of the World Bank, created a number of different platforms – Local Public-Private Stakeholder Committees, and Government-Contractor Forums – that brought local government officials together with local private sector firms in order to identify ways to improve contracting outcomes. National forums enabled munic- ipalities to learn from each other. A linked effort also increased community monitoring of procurement transactions and contract implementation, in order to increase transparency and ensure that communities achieved the full benefits from procurement. 23 A useful approach to improving municipal procurement outcomes in Georgia is likely to feature many of these actions. The transparency of procurement operations is likely to be increased by steps Georgia is taking to adopt the Open Contracting Data Standards. Making Improving Efficiency in Public Procurement in Georgia full use of that information to drive better performance will probably require the creation of mechanisms for government, private sector, and civil society to find ways to solve local problems that are currently harming procurement outcomes and the people who rely on public services in their daily lives. RECOMMENDATION The Government of Georgia should undertake a program to improve procurement perfor- mance among municipalities. The focus of the program should be to provide assistance to poorly performing municipalities to adopt practices and behaviours found to have con- tributed to the better outcomes achieved by well performing municipalities. Elements of a well-designed program of support will include capacity building, increased public interaction among government, potential contractors, and civil society, combined with enhanced monitoring that makes use of the increased access to information from the adop- tion of Open Contracting Data Standards. The SPA should be given primary responsibility for identifying municipalities that are under-performing and creating capacity building programs to support improvement. Responsibility for improvement of performance should rest primarily with the individual municipalities. The Government of Georgia may want to consider establishing a yearly review of municipal performance to be produced by the SPA and other relevant government agencies and creating an annual process in which municipalities set performance improvement targets. 24 VIII Improving Procurement Outcomes Improving Efficiency in Public Procurement in Georgia Increasing Participation by Small and Medium-Sized Enterprises U ntil now, our discussion of improving performance has focused exclusively on changing the practices of public sector organizations. Improvement in outcomes could be measured by increases in the proportion of tenders that were successful, or the amount of money saved by the government. Public procurement outcomes could also be improved by increasing the extent to which procurement spending generates private sector growth and jobs. This dimension emphasizes a second objective of public spending through procurement – the objective of using public spending to catalyse growth. Assessing the impact of procurement on private sector growth could potentially be ap- proached in a variety of ways. One possible effort could be to attempt to calculate the im- pact on a firm of winning a state contract, in regard to increases in revenues, profitability, investment, and employment. Such a far ranging analysis requires access to data sources that are outside the scope of the current study. Instead, this study did an initial analysis focused on the extent to which private sector firms participated in procurement competitions and were awarded contracts. Small firms participate in public procurement tenders at a substantially higher rate than either me- dium or large sized firms. See Figure 16. There appears to be little difference in the rate of success of small, medium, and large firms when they bid. See Figure 17. 25 Figure 16. Bidding of firms by year and by size of firm 45% Large Improving Efficiency in Public Procurement in Georgia Medium of this size that submitted bids 40% Small Percentage of firms 35% 30% 25% 20% 2013 2014 2015 2016 Figure 17. Success rate and number of bidders by size of firm 1.00 0.75 Success rate 0.50 0.25 Large Medium 0.00 Small 4 8 12 Number of bidders Overall, small-sized firms obtain 53.2% of the total number of contracts, capturing the highest proportion of contracts and value in procurement tenders associated with municipalities, and the lowest proportion in tenders by ministries and state owned enterprises. Small firms capture less than 20% of the value of purchases by ministries and state owned enterprises. See Table 2. Increasing the proportion of contracts won by small and medium sized firms is strongly linked to changing outcomes in ministry and state-owned enterprise procurement. 26 Table 1. Contract awards, by size of firm Contracts won Contracts won (% of transactions) (% of value) Improving Efficiency in Public Procurement in Georgia Small 53.2 24.2 Medium 27.1 33.8 Large 19.7 42.0 Table 2. Contract awards, by size and type of procuring entity (%) Small Medium Large by volume by value by volume by value by volume by value Educational facility 60.3 47.7 25.1 34.9 14.6 17.4 Ministry 42.0 17.0 28.6 33.6 29.5 49.4 Municipality 63.4 33.7 22.0 35.0 14.6 31.4 State-owned Enterprise 42.5 19.9 32.7 31.4 24.7 48.8 Other 48.9 24.1 29.9 35.0 21.1 40.9 The World Bank has an extensive program of work with the Government of Georgia on private sector development. Work on making best use of procurement spending to drive business development should complement and be connected to this broader effort. Ef- forts to strengthen the participation of small and medium firms in procurement exist in many countries. Preferential programs for enhancing participation of SMEs by altering the selection and award processes in favor of SMEs are the most widely used interventions. Set-asides, price preference margins, or offset requirements relating to SME inputs or subcontracting, or a combination of the above are the most commonly used methods. In addition, some economies also make efforts to build the competitiveness of SMEs; provid- ing financial flows for implementing contracts; and sharing opportunities for participation and facilitating partnerships among bidders. Restricting competition for certain items, commodities or procurement below a value threshold for SME sector is practiced very widely across many economies – USA, Brazil, India, China, South Africa, to name a few.19 Many countries also undertake efforts to build the capacity of the SME sector to be more competitive in public procurement contracts. Brazil’s example of SEBRAE20, an institution supporting the development of small and micro companies in Brazil, is a classic example of such an effort. Countries have taken steps such as waiving bid security or modifying contracts to allow for higher advances to lower the costs of participation for SMEs and increase their ability to perform contracts. While these initiatives can and have yielded 27 positive results, they must be managed carefully in order to prevent abuse. The current analysis just begins to scratch the surface of this important issue. Work is needed to gain a better understanding of the factors that influence the decision of firms to register in the public procurement system. A second set of questions relates to the fac- Improving Efficiency in Public Procurement in Georgia tors that influence the decision to bid, especially given the existence of a number of clear preferences. A third set of questions relates to the benefits derived from being awarded a state contract. Answering these questions and identifying ways to enhance the impact of procurement spending on private sector development will require collaboration among a number of different contracting authorities/government agencies in order to obtain the necessary data. RECOMMENDATION The Government of Georgia should prioritize efforts to strengthen the impact of procure- ment spending on private sector development. In these efforts, the Government of Georgia can consider affirmative steps such as creating set asides for SMEs, modifying bid security requirements, and changing the timing of payments as part of this effort. The SPA should have primary responsibility for implementing this recommendation, working in close collaboration with government contracting authorities/agencies involved in small and medium enterprise development. The SPA should monitor SME participation rates and the proportion of contracts awarded to SMEs, especially for ministerial and state-owned enterprise transactions. 28 IX Improving Outcomes Improving Efficiency in Public Procurement in Georgia Improving Data Collection and Analysis M easuring procurement performance is an integral dimension of the successful implementation of reform. The ability to track performance over time allows for adaptations in policies, practices, and individual behaviour and is instru- mental in improving outcomes. The analysis that has been presented in this report demonstrates the potential value of examining transaction-level information on procurement generated and captured by ex- isting systems. Data analysis could be a much more valuable tool if existing data sources were expanded to include information on contract implementation, as well as on the use and maintenance of state assets. A second critical expansion would be to develop processes that enable two-way exchanges of information – between state and non-state parties. Both the private sector and citizens/ users of public services have access to information on procurement that would enable a more complete perspective on performance and factors influencing procurement. At the same time, modern technology is continuing to expand possibilities for information shar- ing across groups. A third dimension of establishing an effective data analysis function is to identify methods of presenting analysis that enables policy makers and other stakeholders to make use of the analysis. Reports and figures that do not influence the dialogue about procurement or the decisions that are made are of limited value. The Government of Georgia is already taking important steps to make better use of procure- ment data. Adopting the Open Contracting Data Standard not only commits the Government to sharing information, but also to structuring data in ways that allow for its use by others. Continued progress in implementing Open Contracting Data Standards will be critical. In addition to the steps that are already underway, a program of work designed to im- prove outcomes through strengthened data analytics would involve efforts to: (1) build analytical capacity; (2) enhance the depth of data resources; (3) establish platforms for data exchanges, with a particular focus on data exchanges that feed directly into policy making; and (4) create formats for regular reporting on data findings, and a mechanism to adapt the formats to better inform policy makers and other stakeholders. 29 RECOMMENDATION The Government of Georgia should undertake a program to improve procurement out- Improving Efficiency in Public Procurement in Georgia comes by strengthening its data analytics function, with a focus on data analysis in the State Procurement Agency. The Data Analytics program can assist in providing policy makers and other stakeholders with continuous information on the progress of reform. The critical first step in the program is the implementation of the Open Contracting Data Standard. The SPA may wish to establish an annual report on its progress in data analytics, with a specific focus on the extent to which data analytics findings have influenced policies, practices, behaviors, and outcomes. 30 X Conclusion Improving Efficiency in Public Procurement in Georgia T his study examined the performance of the Georgian procurement system using data on transactions covering the period 2013-2016. Overall, the study found that the system functions at a high level of effectiveness. Nevertheless, it also identi- fied some shortcomings and weaknesses in the way procurement is conducted in Georgia. Addressing these weaknesses has the potential to deliver substantial improvements in the efficiency and value for money obtained through the procurement process. Going forward, it is recommended that the Government of Georgia undertake a program of work to: I) reduce the number of small contracts; improve the level of competition in large works contracts, particularly in the top ten II)  procuring entities; III) improve performance procurement among the poorest-performing municipalities; IV) strengthen the impact of procurement spending on private sector development. In achieving these four goals, a fifth is also important: strengthening the analytics function within the Government of Georgia, allowing pro- V)  active and regular monitoring of progress against relevant indicators. Topics for future analysis might include examining the frequency of repeated contract awards to the same supplier, and the extent to which contracts are shared among a supplier groups. The report has provided some specific recommendations on how to begin tackling some of these challenges. International experience can provide a guide, but the most appropriate response can best be determined by the Government of Georgia. 31 XI Bibliography Improving Efficiency in Public Procurement in Georgia The Brazil Business (2017) – Introduction to SEBRAE. Accessed at: http://bit.ly/2HkLIsh Estache, Antonio and Atsushi Iimi (2008) – “Procurement Efficiency for Infrastructure Development and Financial Needs Reassed.?” World Bank Policy Research Working Paper No. 4662, July 2008. Accessed at: http://bit.ly/2Ho4Am4 European Commission (2016) – “Public Procurement Indicators”. DG GROW G4 – Innovation and e-Procurement, Dec 19, 2016. Accessed at: http://bit.ly/2qQKn21 G20 Global Platform on Inclusive Business (2016) – “Policy Case Studies on Inclusive Business: Preferential Public Procurement”. Accessed at: http://bit.ly/2JjMxhD Institute for Development of Freedom of Information (2017) – Implementation Assessment of the Georgia Public Procurement Legislation. Accessed at: http://bit.ly/2vBL44n McVay (2017) – Getting Started in Federal Contracting. Sixth Edition. Chapter 11. Accessed at: http://bit.ly/2vz37rC Ngwenya and Ramotsamai (2017) – South Africa: Revised Preferential Procurement Regulations, 2017. Accessed at: http://bit.ly/2qUKE39 OECD (2015) – Government at a Glance. Accessed at: http://bit.ly/2qSzblD 32 Endnotes Improving Efficiency in Public Procurement in Georgia Report delivered by the World Bank in collaboration with the UK Government/UK’s Good Governance Fund. The authors acknowledge the support, input, and valuable contributions from Department for International Development (DFID), Good Governance Fund, British Embassy in Tbilisi and officials from the State Procurement Agency of Georgia. 1. The State Procurement Agency of Georgia was originally established in 2001. The Agency went through a number of organizational permutations until it was restructured and reformed by the 2014 act. 2. Figures are averages for the period 2013-16. 3. The World Bank has also undertaken an analysis of the Georgia e-procurement system which provides some complementary information regarding the efficiency of procurement processes. 4. See: Institute for Development of Freedom of Information (2017), for an up-to-date discussion of the dif- ferent mechanisms for undertaking public procurement. 5. Transactions that are excluded from the Public Procurement Law are not captured in the SPA database. While the vast majority of the public procurement transactions are covered, we are unable to quantify the precise scale and value of excluded transactions. Future studies may wish to explore this issue further. 6. As a point of comparison, the OECD calculated that procurement as a percentage of government expenditure averaged 29% in 2013. See: OECD (2015) 7. This figure is slightly below the 13.8% average for EU countries in 2015. See: European Commission (2016). The importance of procurement spending in GDP reflects the role of the state in many advanced economies. 8. While there currently is no standardized approach to describing procurement system performance, the studies that do exist frequently focus on a range of variables relating to the procurement process, such as various aspects relating to the time it takes to undertake a procurement, or the time it takes from the award of contract to the signing of a contract. The emphasis on these variables reflects the existence of serious and systemic procedural bottlenecks in many countries that prevent their procurement systems from fulfilling its core function. This study has not undertaken rigorous analysis of the efficiency of the various steps in the procurement process, such as the average time that is given for advertisement or the percentage of contracts that were awarded during the last month of the fiscal year, since these procedural issues were not at the forefront of the Government’s performance concerns. Georgia’s success in creating a public procurement system that is able to execute a large number of transactions and deliver required goods, services and works within a given fiscal year is a tremendous achievement, and stands in stark contrast to the struggles many countries have encountered in undertaking procurement reform. The e-procurement assessment that the World Bank is undertaking will provide a robust analysis of the efficiency of various phases of procurement. 9. Failed tenders can result from an absence of bidders, the disqualification of all bids during the bid review process, or the inability of the selected bidder and the contracting authority to sign a contract. 10. Evidence on the optimal number of bids is clearest for infrastructure contracts. See, for example: Estache, Antonio and Atsushi Iimi (2008) 33 11. Failed tenders are defined as tenders where no bids were received, the tender was cancelled, or the contract not awarded. 12. The research did not explicitly examine differences in failure rates by CPV code. Improving Efficiency in Public Procurement in Georgia 13. Tenders for large contracts generate 2.2% greater savings as compared to smaller contracts, as determined by contract cost versus the estimated cost. The 2.2% savings translated into a total savings of 1,738,933 GEL. Combining 30% of the smallest contracts to create contracts valued at 200,000 GEL would reduce the number of contracts by 27,032. Tenders for contracts valued 200,000 GEL and above fail at a rate of 18.8%, while tenders for contracts below 200,000 GEL fail at a rate of 29.8%. 14. A Framework Agreement is an agreement between one or more contracting authorities and one or more economic operators, the purpose of which is to establish the terms governing contracts to be awarded during a given period, in particular with regard to price and, where appropriate, the quantity envisaged. Framework agreements provide an efficient and readily available way for contracting authorities to source works, goods and services. 15. The State Procurement Agency (SPA) of Georgia has utilized framework agreements for eight products. The effort was initiated in 2011, started with an agreement for only petroleum, however gradually the list ex- panded, which now includes: mobile service, paper, desktop/laptop, tires, UPS (unlimited power supplies), printers and broadcasting (advertising time). 16. The SPA has also put together an eMarket module which is part of the procurement system. It is intended to provide information regarding prices to help procuring entities in their market research. Although useful, the eMarket module can be difficult to maintain as prices fluctuate often in the Georgian market place. Looking forward SPA could utilize their framework agreements into an eCatalogue module of the e-procurement system to better streamline the process for contracting authorizes of purchasing commonly procured items. 17. 369,272,185 GEL would be saved if each of the top 10 procuring entities achieved the same level of cost savings as the current top performer. 18. Georgia is often identified as one of the world’s leaders on reducing corruption. The country ranked 44 on the latest TI Corruption Perception Index – an enormous improvement from 2003 when it was identified as being the most corrupt country in Eurasia. 19. For more information on these cases, see: G20 Global Platform on Inclusive Business (2016), for Germany, Rep. of Korea, India, South Africa, United Kingdom; McVay (2017), for USA; The Brazil Business (2017), for Brazil; Ngwenya and Ramotsamai (2017), for South Africa. 20. Serviço Brasileiro de Apoio às Micro e Pequenas Empresas (Brazilian service of assistance to micro and small enterprises) 34 Improving Efficiency in Public Procurement in Georgia Findings from Data Analysis of Public Procurement Transactions in Georgia 2013-2016