ICRR 14037 Report Number : ICRR14037 IEG ICR Review Independent Evaluation Group 1. Project Data : Date Posted : 03/13/2013 Country : Turkey Is this Review for a Programmatic Series? Yes No How many operations were planned for the 4 series? How many were approved? 4 Series ID : S123073 First Project ID : P071052 Appraisal Actual Project Name : Programmatic Public US$M ): Project Costs (US$M): $500 $518.62 Sector Development Policy Loan (ppdpl) L/C Number : L7394 Loan/ Loan US$M): /Credit (US$M ): $500 $518.62 Sector Board : Economic Policy US$M): Cofinancing (US$M ): Cofinanciers : Board Approval Date : 06/29/2006 Closing Date : 09/30/2007 09/30/2007 Sector (s): Central government administration (40%); Compulsory pension and unemployment insurance (30%); Sub-national government administration (15%); Health (10%); Compulsory health finance (5%) Theme (s): Law reform (25% - P); Public expenditure; financial management and procurement (25% - P); Social risk mitigation (24% - P); Municipal finance (13% - S); Health system performance (13% - S) Second Project ID :P088837 Appraisal Actual Project Name : Second Programmatic Project Costs (US$M): US$M ): $400.0 402.2 Public Sector Development Policy Loan (ppdpl 2) L/C Number : L7553 Loan/ Loan US$M): /Credit (US$M ): Sector Board : Economic Policy US$M): Cofinancing (US$M ): Board Approval Date : 06/19/2008 Cofinancers : Closing Date : 06/30/2009 06/30/2009 Sector (s): Central government administration (35%), Law and justice (25%), Sub-national government administration (20%), Other social services (10%), Health (10%) Theme (s): Judicial and other dispute resolution mechanisms (29% - P), Public expenditure, financial management and procurement (29% - P), Health system performance (14% - S), Social safety nets (14% - S), Other public sector governance (14% - S) Third Project ID :P112495 Appraisal Actual Project Name : Restoring Equitable Project Costs ( US$M): US$M ): $1,300 $1,260.25 Growth And Employment Programmatic Development Policy Loan L/C Number : Loan/ Loan US$M): /Credit (US$M ): Sector Board : Economic Policy US$M): Cofinancing (US$M ): Board Approval Date : 03/23/2010 Cofinancers : Closing Date : 11/01/2010 11/01/2010 Sector (s): General public administration sector (30%), Other social services (20%), Health (20%), General education sector (20%), Banking (10%) Theme (s): Health system performance (25% - S), Debt management and fiscal sustainability (25% - P), Improving labor markets (24% - S), Education for all (13% - P), Social safety nets (13% - S) Fourth Project ID :P123073 Appraisal Actual Project Name : Second Restoring US$M ): Project Costs (US$M): $700 $736.61 Equitable Growth And Employment Programmatic Development Policy Loan L/C Number : Loan/ Loan US$M): /Credit (US$M ): Sector Board : Economic Policy US$M): Cofinancing (US$M ): Board Approval Date : 05/05/2011 Cofinancers : Closing Date : 12/31/2011 12/31/2011 Sector (s): General public administration sector (30%), Other social services (20%), Health (20%), General education sector (20%), Banking (10%) Theme (s): Health system performance (25%), Debt management and fiscal sustainability (25%), Improving labor markets (24%), Education for all (13%), Social safety nets (13%) Evaluator : Panel Reviewer : ICR Review Group : Coordinator : Ismail Dalla Michael R. Lav Mark Sundberg IEGPS2 2. Project Objectives and Components: a. Objectives: Background : The Implementation Completion Report (ICR) evaluated two series of programmatic development policy loans (DPLs) with the following objectives. The first series consisted of Programmatic Public Sector Development Policy Loans 1 and 2 (PPDPL 1 and 2). The objectives of this first series were : (a) sustaining the current enabling macroeconomic framework; (b) reforming the country’s social protection system which consists of social security, health insurance and social assistance; (c) continuing the on-going process of upgrading the public financial management and budget reforms; and (d) improving the administration and governance of the public sector (Para 86, page 29 of the PD for PPDPL1). These objectives were maintained for PPDPL 2 with slight modifications (paragraph 70, page 18 of the PD). The second series was comprised of the Restoring Equity Growth and Employment Programmatic Development Policy Loans 1 and 2 (REGE- DPL 1 and 2). The objectives of the REGE-DPL series were to REGE -DPL1 improve: (1) economic management; (2) public financial management; (3) affordable universal health care and improved educational access; (4) employment; (5) improved investment climate; (6) increased financial intermediation, especially to SMEs (PD, page i). ). In addition, the REGE-DPL continued to support public sector REGE -DPL2 reforms including in social security and public financial management . The objectives of the REGE- DPL 2 were to support equitable growth and job creation over the medium term, by advancing critical business climate and broader competitiveness reforms, while carrying forward Turkey ’s public expenditure reform agenda with a focus on the delivery of inclusive social programs at sustainable cost . It also included reforms in the area of equitable growth and employment, a focus on the business climate and selected improvements to vocational and basic education (PD page ii). While their objectives were distinct, the ICR covered both the PPDPL and REGE -DPL series in a single evaluation. For the purposes of this ICR Review and consistency with the OPCS /IEG Harmonized Evaluation Criteria, IEG first evaluates each of these series separately, then combines the ratings for relevance and efficacy to correspond with the ICR’s structure. b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: The policy areas for PPDPL1 and 2 included: (1) sustaining the current enabling macroeconomic framework (2) reforming the country's social protection system (social security, health insurance, and social assistance ) (3) continuing the on-going process of upgrading the public financial management and budget reforms (4) improving the administration and governance of the public sector The policy areas for REGE-DPL1 and 2 were different from the above. They included: (1) fiscal management (2) equitable growth and employment (a) affordable universal health care and improved educational access (b) employment (c) increased financial intermediation, especially to SMEs . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: PPDPL1 was approved in June 2006 in the amount of EUR 403 million (US$500 million) and was declared effective on August 16, 2006. Disbursements amounted to $518.62 due to the appreciation of Euro . The loan was closed on schedule on September 30, 2007. PPDPL2 was approved in June 2008 in the amount of EUR 255.4 million (US$400 million) and declared effective on July 23, 2008. Disbursements amounted to US$ 402.2 million. The loan was closed on schedule on June 30, 2009. REGE-DPL1 was approved in 2010 in the amount of EUR 931 (US$1.3 billion) and declared effective on April 9, 2010. Disbursements amounted to $1.260.25 billion. The loan was closed on schedule on November 1, 2010. Loan maturity was 19.5 years including a grace period of 16 years. REGE-DPL2 was approved in May 2011 in the amount of EUR 506.1 million (US$ 700 million). Disbursements amounted to $736.61 million. The loan was closed on schedule on December 31, 2011. REGE-DPL 2 has a maturity of 26.5 year maturity and 9 years of grace period with level repayment of principal. 3. Relevance of Objectives & Design: a. Relevance of Objectives: IEG assessed the relevance of objectives was based on the following considerations : Over the 2000s, Turkey had made significant progress in restructuring its economy and financial sector . Turkey has experienced rapid economic growth and its financial markets and banking system have performed well. The overall level of government debt was reduced on account of fiscal management reforms and economic expansion. The substantially relevant PPDPL series initially sought to consolidate these gains by promoting macroeconomic stability and strengthening the social security and public financial management (PFM) systems. With the second operation, the objectives had broadened to include further deepening of PFM reforms, regional development, justice system reform, and management of the fiscal costs of the health and pensions systems, the bulk of which were consistent with Turkey ’s priorities and needs at the time (ICR, p. 18) By the close of the PPDPL series, Turkey was contending with the economic and social impact of the global financial crisis. The shorter term focus of the REGE-DPL1 – on domestic fiscal and external debt sustainability as well as social assistance – were substantially relevant to Turkey ’s crisis response needs, although they did detract from the medium-term reform agenda (ICR, p. 18). In other countries, such as Vietnam, the Bank opted to support a separate crisis response DPL in parallel in order to minimize the momentum of longer term policy and institutional reforms. Nevertheless, the REGE-DPL2 refocused attention on the longer term, structural concerns in fiscal management, competitiveness and employment . The objectives of the PLPDPL and REGE -DPL series were both consistent with the Bank ’s 2008-11 Country Partnership Strategy (CPS), which aimed to help Turkey realize its medium term goals of rapid and sustained growth with equity. Both series were broadly aligned with the following CPS pillars – (i) improved competitiveness and employment, (ii) equitable human and social development, and (iii) efficient provision of high quality public services . The 2008 CPS was in turn was developed concurrently – and therefore reflected – the priorities of Turkey’s Ninth Development Plan. Both of these series remain relevant to key elements of the 2012-15 CPS, which maintains a focus on competitiveness and employment, as well as equitable service delivery . The current CPS, however, has gone beyond these four DPLs and now includes focus on sustainable development (through the use of renewable energy, environment management, and urban development ). On balance, the relevance of objectives of both series is substantial . b. Relevance of Design: In assessing the relevance of design of both series, IEG identified important strengths : For both series, the causal linkages between the intended outcomes and specific policy areas were robust . In the case of the PPDPL series, the selection of policy areas were appropriately sequenced measures to promote sound macroeconomic management and fiscal consolidation, followed by efforts to strengthen public management and improve service delivery . Similarly, the REGE-DPL series continued to support Turkey ’s evolving national goals, even though REGE -DPL1 was diverted from medium-term priorities as a result of the crisis. It sought to maintain the focus on improvement fiscal and public sector management, while simultaneously supporting measures to enhance the investment climate, competitiveness, and employment . Both series were broadly based on sound analytical underpinnings , including a wide array of Bank AAA including a series of Country Economic Memoranda (from 2003-8), various fiduciary assessments (dating back till the early 2000s), an investment climate assessment (2010), financial sector assessments (2007 and 2011), and various social sector assessments (ICR, p. 15-6). The ICR also notes that the AAA was undertaken with the ownership of the Government and relevant counterpart agencies . However, important caveats are noted below. Despite these strengths, this review also notes the following shortcomings : The Bank supported the competitiveness and employment agenda with a separate CEDPL series, implemented parallel to the PPDLP series. IEG’s June 2009 ICR review concluded that the CEDPL achieved a satisfactory outcome – albeit at exceedingly high supervision costs and with some disappointing results in the area of labor market reforms. While the ICR under review acknowledges “internal Bank� factors for splitting the PPDLP and CEDLP series, it does not adequately address the counterfactual – combining these series to address competitiveness and public sector reforms together while economizing on Bank supervision costs (ICR, p. 16). In fact, such an approach is reflected in the design of REGE -DPL series, which appears to have managed the risks of complexity . The PPDPL series benefitted from a robust and stable macroeconomic framework, which was supported by an IMF program (ICR, p. 14). The same cannot be said for the REGE -DPL series, which was forced to contend with the impact of the global financial crisis in the absence of an IMF program . While the Program Documents for REGE-DPL series confirm that the Bank and Fund had similar assessments on macroeconomic management and structural reforms (p. 34, REG-DPL1 PD), concerns about macroeconomic vulnerability remained through the 2009-2011 period. In addition to a flat saving rate, the current account deficit (CAD) -- financed largely by short-term external capital flows – deteriorated from 6.4 percent in 2010 to 10.0 percent in 2011 (p. 15, ICR). In keeping with the post-crisis experience of other Bank borrowers that did not have IMF programs, a more formal agreement on the country ’s macroeconomic framework would strengthened the “confidence-building� objectives of the Turkey’s central bankers. For both series, the design of reforms across different policy areas was mixed, and was ultimately dependent on the depth of AAA and success of institutional strengthening financed by investment projects (ICR, p. 18-9). In some sectors such as health, the underlying analytical and capacity building efforts were robust . In others, these building blocks as well as political economy analyses were found to be lacking (for example, decentralization and public administration reform ). Finally, both series lacked a robust and systematic results framework – a weakness also identified in IEG ’s ICR review of the CEDPL. This weakness was further reflected in the absence of “systematic monitoring of outcomes across a range of sectors ." Section 10 elaborates on M&E issues. IEG assessed the design of the PPDLP series as substantial based on the strengths of its causal linkages, solid analytical underpinnings, and sound macroeconomic framework . Its design was hampered in part by the lack of a clear rationale for supporting the competitiveness agenda through a separate operation, and varied quality of design of specific actions and the results framework . REGE -DLP series shared the same design strengths and some weaknesses, its distinctive While the REGE- challenges related to (i) difficulties in balancing longer term structural reforms with short -term crisis response, and (ii) the need for clearer agreement on the macroeconomic framework (in particular the deteriorating CAD). Relevance of design for the second series is rated modest . In concurrence with the ICR, IEG’s combined assessment of relevance of design for both series is modest . 4. Achievement of Objectives (Efficacy): The objectives of the PPDPL series are: (A) Sustaining an enabling macroeconomic framework The PPDPL series, implemented from 2006 through 2008, benefited from a robust macroeconomic framework and an active IMF program. The program, which envisioned low single digit inflation and a 10 percentage point reduction in the Government’s net debt to GDP ratio, was successfully completed . However, there have been no follow up operations with the IMF since 2008. Achievements over the course of the implementation of the first series are further elaborated under “Macroeconomic Developments� below. The achievement of this objective is rated substantial . (B) Social security and social assistance reform Pension system reforms supported by the series sought to ensure long -term financial viability. The Social Security and Universal Health Insurance Law was enacted in June 2006 and amended in May 2008. The operation then supported the establishment of a Universal Health Insurance (UHI) system and universal health coverage has been achieved with improvements in management of health sector expenditures . In addition, the three components of the public pension system were combined into a single social protection system in order to better target social insurance to those in need, while increasing contributions and reducing expenditures . A target of the series was to maintain the combined social security deficit below 4.5 percent of GDP to ensure its long term sustainability (p. 23, ICR). The deficit was estimated at 3.9 percent in 2010. A private pension system has also been created as a supplemental insurance . The private pension fund has grown rapidly but remains relatively small accounting for less than 2 percent of GDP. Turkey suffered from a low savings rate (13 percent of GDP in 2011) which was addressed in part through social security reforms . high . The achievement of this objective is rated as high. (C) Public financial and expenditure management reform Reforms to achieve this objective supported modernization of public administration, especially upgrading of auditing standards and the use of the medium term framework for budgeting . Improved revenue and expenditure management contributed to better public financial management and management of public debt . A central government budget law consistent with the Medium Term Financial Plan of the Government was passed in 2006 but remains a long term agenda. For instance, medium-term budgeting was progressively institutionalized under the series, particularly in sectors such as health, yet the degree to which medium term expenditure management took root was varied across Government . Overall cash flow management of Government improved, and Turkey strengthened its debt management office . There were no specific indicators on this subject in the ICR . Also, it is important to note the Bank’s technical assistance – in addition to this particular DPL series – is credited with furthering Turkey’s PFM efforts. The achievement of this objective is rated as substantial . (D) Public administration and governance reform In 2003, Turkey initiated a comprehensive public administration reform process with the objective of establishing a more participatory, transparent, and public sector which provide high quality public services in a timely, efficient and equitable manner. The PPDPL series sought to further this objective by supporting decentralization and the establishment of regional Development Agencies (DAs) with legal access to funding from various higher levels of government. While initial efforts were delayed (and this set of initiatives were dropped from subsequent DPLs), 26 regions have established DAs and are applying for various capital projects based on development plans. Beyond these structures, however, the actual performance of these DAs remains to be assessed systematically. In addition, the second operation in the series also sought to improve the speed and functioning of the justice system. Regional Courts of Appeal were established to reduce the backlogs of cases within the Court of Appeals. On-line connections among courts have also been established through implementation of the judicial network project. In the absence of specific indicators against baselines, the success of regional development and justice system reform efforts is difficult to assess . The achievement of this objective is rated as modest . REGE- REGE -DPLs (E) Fiscal Management Under the REGE-DPL series, the Government deepened public financial management reforms and strengthened social programs at a sustainable cost to maintain universal health coverage and to protect the long-run financial sustainability of the health and social security system . As noted above, the social security reform was implemented through secondary legislation and regulation, and the merger of the three previously self-standing social security systems . There was a perceived need for continued implementation of the unified social security reform and cost containment in the health sector (ICR, page 9 and 10). Under REGE-DPLs, a global budget for Ministry of Health ’s hospitals and spending limits for university and private hospitals were enforced to better manage hospital expenditures . These measures generally helped further the REGE -DPL series objective of increasing the value for money in the delivery of public services . However, more detailed evidence in the ICR would have enabled IEG to confirm actual improvements in output efficiency . The achievement under this objective is rated as substantial . (F) Equitable growth and employment Under the REGE-DPL1 measures included an increase in short -term unemployment compensation by 50 percent and its extension from 3 to 12 months, extension of credit guarantees to SMEs, and vocational training and hiring of 15,000 new preschool teachers. The unemployment rate started to decline sharply since late 2009, initially due to domestic factors including import intensive domestic consumption, and then subsequently in 2012, due to export growth and diversification . As the employment situation improvement, the Government was able to re -focus its efforts on long-run reforms – not only the above-mentioned measures on health and social security modernization, but also those relating to the business climate. Specifically, REGE-DPL2 was reformulated to include various “confidence building� measures to address continued heightened uncertainty in the financial markets . These included the Central Bank’s decision to act as the blind broker in liquidity provision between private banks, guaranteeing short -term loans. This measure was designed to remove counter -party risks between financial institutions . A second measure, approved in 2009, included credit guarantees for SMEs . Ultimately, both of these shorter term measures were not as critical as originally envisaged, and could have been more effectively withdrawn as medium-term competitiveness measures were introduced . By 2010, Turkey experienced a credit expansion on account of considerable capital inflows into the banking system and Istanbul Stock Exchange, as well as Bank-financed SME loans. Lending to SMEs by the banking system rose sharply and accounted for 23 percent of outstanding loans in the system by the end of 2011. In parallel, the Commercial Code was adopted to spur private sector development, although implementation will require capacity enhancements in the auditing profession. Evidence that measures under this policy area strengthened Turkey ’s investment climate is partial. It is not possible for IEG to attribute measures supported by the series (for example, a reduction in social security contributions) to employment growth. Also Turkey’s investment climate relative to other comparators has yet to st register significant improvements (for instance, according to Doing Business, Turkey ranked 71 in 2012, only a rd slight improvement from 73 in 2011. The level of foreign direct investment remains below crisis levels . In addition to vulnerabilities to a volatile global economic environment, structural reforms in the financial and private sectors will likely need to be deepened to ensure resilience . The achievement under this objective is rated as modest . 2008 -2011) Macroeconomic Development in Turkey (2008- 2011 ) Turkey’s key economic indicators over the course of both DPL series are shown in the table below . Turkey's GDP growth slowed dramatically in 2008, and declined by 4.8 percent in 2009. However, in 2010, the economy rebounded sharply and grew at 9% putting output well above pre-crisis levels. The rebound greatly benefitted from the general recovery in the global economy (especially Turkey's major trading partners including Germany ) and expansionary macroeconomic policies that caused a surge in credit -financed, import-intensive domestic demand. The latter, however, caused a sharp increase in the current account deficit (CAD) -- largely financed by short-term debt and other volatile flows (portfolio investments). Foreign direct investment has not recovered to pre-crisis levels and amounted to only 1.1 percent of GDP in 2010. Given low savings (12.7% of GDP in 2010), Turkey’s economic growth relies on capital inflows to finance imports and capital goods. When inflows are abundant, growth is strong; when flows reverse, the economy contracts, leaving Turkey prone to boom -bust cycles. Since mid-2009, capital inflows have intensified on favorable push and pull factors, including abundant yield -seeking global liquidity, healthy Turkish balance sheets, and Turkish policymakers ’ agile response to the global crisis .(IMF Article IV supplement, January 2012). Even still, the Turkish corporate sector is also exposed to foreign currency denominated debt (Turkey-Corporate Bond Market Development-Priorities and Challenges, World Bank, 2012). Overall, in the view of the Bank's team, Turkey ’s economy is heading for a soft landing . The CAD has fallen considerably since 2011 and is projected to be 7.2 percent of estimated 2013 GDP (IMF Article IV, December 2012). However, annual gross external financing needs remain high in the context of potentially volatile global capital flows. The Government has continued to respond by improving the headline fiscal balance and introducing credit-restraining prudential measures . A key feature is an unconventional monetary policy framework (adjustment of reserve requirements on bank deposits ) intended to deter very short -term inflows, moderate credit growth, and more recently, manage output, exchange rate, and inflation volatility . While it appears to have converged with and contributed to a cooling off of demand, although it will need to be monitored more closely going forward. 5. Efficiency (not applicable to DPLs): 6. Outcome: IEG rated two series separately. The overall outcome for the PPDPL series was rated as satisfactory based on ratings for relevance of objectives (substantial), relevance of design (substantial) and achievement ratings for four objectives -- macroeconomic framework (substantial), social security (high), public financial management REGE -DPL series was rated moderately (high), and public administration (modest). The outcome for the REGE- satisfactory . This was based on ratings for relevance of objectives (substantial), relevance of design (modest), and achievement ratings for objectives relating to fiscal management (modest) and equitable growth and employment (modest). IEG rates the overall outcome of the two DPL series as moderately satisfactory . On the macroeconomic front, Turkey remains vulnerable to a potential unfavorable global environment including a recession in Europe. Other concerns include a low level of savings; a low level foreign exchange reserves relative to short-term debt; and the corporate sector's exposure to currency depreciation . Recent assessments suggest that economy is headed for soft landing, the CAD has been reduced, and headline inflation is expected to fall. Government measures to improve the fiscal balance, credit restraining prudential measures, and an unconventional monetary policy appear to be having an effect but will need to be carefully monitored . Ultimately, deeper structural reforms are needed to strengthen the economy ’s long-term resilience in the face of potential shocks in the future . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The program outcomes are subject to economic and external, implementation risks, and political risks . Turkey remains susceptible to macroeconomic risks due to its low savings rate and its dependence on short -term foreign portfolio flows for financing its current account deficit that has increased to an alarmingly high level of 8.5 percent of GDP. Foreign exchange reserves are relatively low and covered only 70 percent of external short-term debt at the end of October 2011. The Central Bank has also recently employed an unconventional monetary policy (changing reserve requirements ) to control capital flows. Implementation of these measures, which is closely watched by international investors, may carry risks of capital reversals . The recent crisis and contraction in European economies creates downside risks to the global demand for Turkish exports (especially from the EU), to output growth and fiscal performance . The capacity to implement key measures such as the newly enacted Commercial Code remains a cause for concern. It will require a major upgrade in the capacity of the auditing profession to deal with the enhanced transparency and accountability requirements introduced by the Code . As in many other countries, the introduction of tighter reporting requirements could bring about requests for delays in the application of these new requirements. Turkey has had a stable government since 2002 under the Justice and Development Party (AKP), facilitating the implementation of major reforms . However, the evolving civil conflict in neighboring Syria may pose additional risks. Overall, the risk to development outcome is rated as significant due to high macroeconomic risks associated with volatile capital flows and evolving European risks . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: Both series were based on a medium -term policy dialogue, strong analytical work, and technical assistance . They were designed with sufficiently flexibility to respond to evolving country development priorities and accommodate changes in economic, social, and political circumstances . The timing of the PPDPL series was in tandem with the launching of the Ninth Development Plan . The substantive policy content and the dialogue underpinning the REGE-DPL continued to focus on these broad objectives . Notwithstanding these strengths, the Bank's performance could have been stronger in the following areas : Bank efforts under REGE-DPL series could have been strengthened with more clarity on the macroeconomic framework. The ICR did not adequately explain how the Bank made trade -offs between shorter crisis response and longer term structural reform goals of the second series, particularly in the absence of an IMF program. More of an emphasis on understanding the political economy factors influencing public sector institutional reforms (particularly PFM, decentralization, and the justice sector ) would have helped improve the focus and likely effectiveness of both series . Also, more vigorous efforts were needed to encourage systematic government monitoring of this complex set of programs. Finally, given that Turkey was the second largest Bank client at the end of 2011, there was need for greater attention to the potential credit risk from a corporate risk perspective . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Most supervision effort for each DPL was combined with the preparation of a subsequent DPL . The supervision was carried out by a field -based country team that maintained a continuous policy dialogue on reforms supported by the DPL series and monitored the agreed outcome indicators . However, the pressures to prepare subsequent operations may have detracted from the quality of supervision reports in the system . Given the absence of a vibrant M&E system, the Bank did not appear to achieve its goal of providing “qualitative inputs� into the process of systematic monitoring by the Government . Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Since 2001, Turkey’s Government has made notable progress in developing a well functioning government bureaucracy with the Deputy Prime Minister for Economic and Financial Affairs providing leadership in the design and conduct of macroeconomic policy and reforms . The Central Bank of Turkey is well respected in the financial community. Turkey’s macroeconomic policy and institutional reforms are coordinated within the Economic Council that included core economic ministers and relevant line ministries in addition to the Treasury. This structure facilitated policy dialogue between the Bank and the government on the two DPL series under review. The government performed well in the areas that there were broad -based support within the country. An example of this is the merger of the three social safety net organizations . Reforms in the private sector such as introduction of the commercial code and creating favorable environment for foreign investment have been more difficult . The two DPL series benefited from a stable government which was in power during the entire implementation period . The current government is committed to pursue necessary reforms that would elevate Turkey to become a major economy in the future . However, the absence of a robust M&E to enable evaluation of key results was missing . Overall, the government performance is carrying out the reforms under the two DPL series was satisfactory . Government Performance Rating : Satisfactory b. Implementing Agency Performance: Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: A major shortcoming of the two DPL series was clarity on the functioning of a country system for monitoring and evaluation (M&E). Both series relied the Treasury Department to carry out systematic monitoring of outcomes across a wide range of sectors, although there was no evidence of such a system being established by the Undersecretary of Treasury (ICR, p. 17). Despite identifying 34 indicators to track the reform program, there were no baseline data for several indicators and some indicators were qualitative and difficult to measure . There is no evidence that a sound M&E has been established by the Undersecretary of Treasury . b. M&E Implementation: As the ICR noted, "while there was a good awareness of where reforms were in terms of the stage of implementation, more needs to be done to monitor arrangements designed and implemented to measure the effects of reforms in terms of outputs and outcomes .� (para 3.3). c. M&E Utilization: This ICR does not provide evidence of how M&E was used in resource allocation or other decisions . M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: None b. Fiduciary Compliance: None c. Unintended Impacts (positive or negative): d. Other: 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Factors relating to the relevance of Satisfactory design, the absence of a robust M&E system, and limited achievements on the investment climate, certain aspects of public administration and governance account for IEG's downgrading of overall project outcomes. Risk to Development Moderate Significant The program outcomes are subject to Outcome : macroeconomic, implementation, and political risks. These are further delineated in Section 7. Bank Performance : Satisfactory Moderately Bank performance during the design Satisfactory and implementation of REGE-DLP series with greater clarity and agreement on the macroeconomic framework. More of an emphasis on the political economy factors influencing public sector institutional reforms (particularly decentralization and the justice sector) may have improved the likelihood of success . Also, more vigorous efforts were needed to encourage systematic government monitoring of this complex set of programs. Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: IEG concurs with the ICR that a flexible approach of DPL series can be helpful in supporting reforms and maintaining dialogue over the longer run . However, in countries such as Turkey where the Bank's exposure is significant, a more focused approach may be needed . Going forward, a focused approach will be all the important given the Bank's limited capital -- an issue raised in IEG's recent evaluation of the Bank's crisis response. Great selectivity places a premium on strong analytical work and ongoing country dialogue . IEG also concurs with the CPSCR: both policy dialogue and attention to M&E are essential for DPL series to achieve their objectives. Equally important are political incentives for Governments to sustain reform efforts over the medium-term. This ICR review also notes that the trade -offs between the shorter crisis response and longer term structural reform goals of policy based operations should be made explicit . Attempting to achieve both objectives within a single series (such as the REGE-DPL) can be especially challenging . Furthermore, DPLs that provide short term external financing in the absence of an IMF program should first seek explicit agreement on the macroeconomic framework and the Government's fiscal stance . 14. Assessment Recommended? Yes No Why? Given their considerable size (US$2.9 billion) and their support for a long term reform agenda, IEG recommends that both series be assessed . A PPAR may also provide a way to better understand how the Bank can support countries seeking to balance short term crisis response needs with longer term structural reform priorities . 15. Comments on Quality of ICR: The ICR covered two different DPL series with a broad of range of objectives . This well-written assessment covered the main achievements of both series as well as some of the challenges posed by the crisis . The report could have been more comprehensive in its treatment of the contextual and design issues -- including alternative approaches -- underpinning the use of the REGE-DPL series as a crisis-response instrument (particularly in the absence of an IMF program). Other economic challenges facing Turkey during 2008- 2010 such as the low saving rate and risks from volatile capital flows deserved more attention . From a corporate risk perspective, the ICR could have usefully addressed the potential risk to the Bank of its large exposure to Turkey . a.Quality of ICR Rating : Satisfactory