Document of The World Bank Report No: ICR2070 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-73200) ON A LOAN IN THE AMOUNT OF EURO 30.0 MILLION (US$ 39.0 MILLION EQUIVALENT) TO THE REPUBLIC OF CROATIA FOR A SCIENCE AND TECHNOLOGY PROJECT February 27, 2012 Private and Financial Sector Development Europe and Central Asia ii CURRENCY EQUIVALENTS (Exchange Rate Effective January 31, 2012) Currency Unit = EUR EUR 1.00 = US$ 1.31 US$ 1.00 = 0.77 EUR FISCAL YEAR January 1 to December 31 ABBREVIATIONS AND ACRONYMS AIP Average Innovation Performance MTR Mid-term Review BI Brodarski Institute NSF National Science Foundation BICRO Business Innovation Centre of Croatia PAD Project Appraisal Document CAS Country Assistance Strategy PDO Project Development Objective CPS Country Partnership Strategy PIP Project Implementation Plan EARTO European Association of Research and PMU Project Management Unit Technology Organizations EC European Commission POC Proof of Concept ECA Europe and Central Asia R&D Research and Development ERA European Research Area RBI Rudjer Boskovic Institute EREF European Regional Economic Forum RDIs Research and Development Institutes EU European Union RI Rudjer Innovations FPs Framework Programs (EU) SEE South East Europe FP7 Seventh Framework Program (EU) SMEs Small and Medium Size Enterprises FY Fiscal Year SPREAD Sponsored Research and Development Program HITRA Croatian Program for Innovative STP Science and Technology Project Technological Development ICR Implementation Completion Report S&T Science and Technology ILO International Labor Organization TTO Technology Transfer Organization IP Intellectual Property TAL 2 Croatia Second Technical Assistance Loan IPA Instrument for Pre-Accession Assistance UKF Unity through Knowledge Fund IRA Institutional Renewal Account UoR University of Rijeka MARIBIC Mariculture Business Innovation Center UoZ University of Zagreb M&E Monitoring and Evaluation ZIE Zagreb Institute of Economics MSES Ministry of Science, Education and Sport Vice President: Philippe H. Le Houerou Country Director: Peter Harrold Sector Manager: Lalit Raina Project Team Leader: Paulo Correa ICR Team Leader: Arabela Aprahamian iii Republic of Croatia Science and Technology Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph Main Text 1. Project Context, Development Objectives and Design ................................................................1 2. Key Factors Affecting Implementation and Outcomes ...............................................................4 3. Assessment of Outcomes .............................................................................................................7 4. Assessment of Risk to Development Outcome ..........................................................................12 5. Assessment of Bank and Borrower Performance ......................................................................13 6. Lessons Learned.........................................................................................................................16 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............................18 Annex 1. Project Costs and Financing ...........................................................................................19 Annex 2. Project Development Objective, Outputs and Results Framework by Component ......20 Annex 3. Economic and Financial Analysis ..................................................................................34 Annex 4. Bank Lending and Implementation Support/Supervision Processes..............................41 Annex 5. Beneficiary Survey Results ............................................................................................43 Annex 6. Stakeholder Workshop Report and Results....................................................................45 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................................46 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................................58 Annex 9. List of Supporting Documents .......................................................................................59 Map iv A. Basic Information Science & Technology Country: Croatia Project Name: Project Project ID: P080258 L/C/TF Number(s): IBRD-73200 ICR Date: 01/30/2012 ICR Type: Core ICR Lending Instrument: SIL Borrower: REPUBLIC OF CROATIA Original Total USD 40.00M Disbursed Amount: USD 41.67M Commitment: Revised Amount: USD 38.47M Environmental Category: C Implementing Agencies: Ministry of Science, Education and Sports Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 12/19/2002 Effectiveness: 05/11/2006 05/11/2006 11/29/2007 07/28/2009 Appraisal: 09/13/2004 Restructuring(s): 12/07/2010 03/31/2011 05/15/2011 Approval: 07/07/2005 Mid-term Review: 06/29/2007 10/24/2008 Closing: 11/30/2009 05/31/2011 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating Potential Problem Project at any No Quality at Entry (QEA): None time (Yes/No): v Problem Project at any time Quality of Supervision No None (Yes/No): (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 15 10 General industry and trade sector 85 90 Theme Code (as % of total Bank financing) Education for the knowledge economy 33 15 Infrastructure services for private sector development 33 35 Micro, Small and Medium Enterprise support 17 40 Technology diffusion 17 10 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Shigeo Katsu Country Director: Peter C. Harrold Anand K. Seth Sector Manager: Lalit Raina Khaled F. Sherif Project Team Leader: Paulo Guilherme Correa Vinod K. Goel ICR Team Leader: Paulo Guilherme Correa ICR Primary Author: Arabela Sena Aprahamian F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The major development objectives of the project are to (i) strengthen and restructure selected research & development institutions to achieve higher levels of applied research, while maintaining their scientific excellence; and (ii) to increase the ability of enterprises to develop, use, adapt and commercialize technology. This would contribute to improving the competitiveness of the Croatian industry both in the domestic and foreign markets. An increased technological base and strengthened linkages between RDIs, scientific community and industry will help enterprises compete more effectively and facilitate Croatia‟s economic integration with the global markets. Moreover, upgrading the technological capabilities of firms would be an important step for Croatia‟s access per se, but also in view of its EU accession. Revised Project Development Objectives (as approved by original approving authority) The revised development objectives of the project are to (i) enable research and development institutions to commercialize research outputs; and (ii) to increase the ability of enterprises, particularly small and medium sized companies to invest in research and development activities (Project Paper July 2009). vi (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Increased number of research contracts with the industry (BI, UKF contracts co- Indicator 1 : financed by business, and the TTOs contracts with business not financed by STP) Value quantitative or 2 30 30* 59 Qualitative) Date achieved 12/31/2006 11/30/2009 05/31/2011 05/31/2011 Comments *Formally revised target was corrected through the ISR (sequence 11) from 30 contracts (incl. % to 55 as to include the target for UKF (25) in addition to the target of 30 of BI achievement) Brodarski Institute. (107% achievement) Increased value (EUR M) of research contracts with the industry (BI, UKF contracts co- Indicator 2 : financed by business, and the TTOs contracts with business not financed by STP) Value quantitative or 0.5 8* 10.9** Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 *Formally revised target introduced as part of the 2009 restructuring to better measure Comments what is measured and refers to BI contracts; (incl. % ** includes the value of UKF contracts (EUR1.9 million) besides the value achieved by achievement) BI (EUR 9 million). (121% achievement) Increased commercialization of R&D results as signed by RI and other TTOs, in terms Indicator 3 : of number of spillovers (including joint ventures) Value quantitative or 0 8 8 5 Qualitative) Date achieved 12/31/2006 12/31/2006 05/31/2011 05/31/2011 Comments There are two new spin-offs in the pipeline, in the areas of biotechnology and (incl. % biomedicine (pharmaceutical), where additional work and funds are being invested to achievement) determine potential applications for commercialization. (63% achievement) Increased commercialization of R&D results as signed by RI and other TTOs, in terms Indicator 4 : of value (EUR million) of spillovers (including joint ventures) Value quantitative or 0 2.8* 3.1 Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments *Formally revised target introduced as part of the 2009 restructuring to better measure (incl. % what is measured. (111% achievement) achievement) Increased commercialization of R&D results as signed by RI and other TTOs, in terms Indicator 5 : of number of licensing agreements signed with business Value quantitative or 0 8 12* Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 vii Comments * includes commercial agreements related to licensing of intellectual property (150% (incl. % achievement) achievement) Increased commercialization of R&D results as signed by RI and other TTOs, in terms Indicator 6 : of value (EUR million) of licensing agreements signed with business Value quantitative or 0 1* 0.75 Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments *Formally revised target introduced as part of the 2009 restructuring to better measure (incl. % what is measured. (75% achievement) achievement) SMEs that have new or significantly improved products new to the market maintained Indicator 7 : (as share of total enterprises): Source Community Innovation Survey Value quantitative or 7.22 7* 10.8 Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments New indicator introduced as part of the 2009 restructuring. (154% achievement) (incl. % achievement) Increased Volume of private R&D mobilized - through programs supported by STP Indicator 8 : (EUR million) (RAZUM, SPREAD and UKF projects with industry) Value quantitative or 6 9.2 13.7 Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments New indicator introduces as part of the 2009 restructuring. (149% achievement) (incl. % achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 : BI's share of revenues from private sector (%)* Value (quantitative 22 35 45 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments *Revised to better measure project activities and objectives. (129% achievement) (incl. % achievement) Indicator 2 : Number of IPs taken to protection from RBI and third parties* Value (quantitative 0** 38** 37 or Qualitative) viii Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised due to changed objective as part of the 2009 restructuring (incl. % ** Restructuring paper of 2009 indicates by mistake 60- error acknowledged in the ISR achievement) (the end target envisaged 5 new IPs); baseline was 33. (97% achievement) Indicator 3 : Number of patents granted to RI* Value (quantitative 0 6 5** or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised due to changed objective as part of the 2009 restructuring (incl. % ** This refers only to core patents approved, though these core patents generated 28 achievement) patent applications approved for specific aspects. (83% achievement) Indicator 4 : Number of IPs transferred to Zagreb and Rijeka's TTOs* Value (quantitative 0 20 18 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised due to changed objective as part of the 2009 restructuring. (90% (incl. % achievement) achievement) Indicator 5 : Number of patents filed to the TTOs (cumulative) Value (quantitative 0 8 8 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised due to changed objective as part of the 2009 restructuring. (100% (incl. % achievement) achievement) Indicator 6 : Amount of financing (EUR million) provided - RAZUM program* Value (quantitative 18.03 33.68 34.2 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised as part of the 2009 restructuring following removal of Venture Capital (incl. % activity. (102% achievement) achievement) Indicator 7 : Amount of financing (EUR million) provided - SPREAD program* Value (quantitative 0 1.5 2.07 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised as part of the 2009 restructuring following removal of Venture Capital (incl. % activity. (138% achievement) achievement) Indicator 8 : Amount of financing (EUR million) provided - TECHRO program* Value (quantitative 0 12 13 or Qualitative) ix Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised as part of the 2009 restructuring following removal of Venture Capital (incl. % activity. (108% achievement) achievement) Indicator 9 : Number of SMEs directly supported by programs* Value (quantitative 37 78 89 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments * Revised as part of the 2009 restructuring following removal of Venture Capital (incl. % activity. (114% achievement) achievement) Indicator 10 : Number of projects supported by UKF (total) Value (quantitative 0 50 91 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments 182% achievement (incl. % achievement) Indicator 11 : Number of projects supported by UKF involving cooperation with industry* Value (quantitative 0 15* 28 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments *new indicator and target added during 2009 restructuring. (187% achievement) (incl. % achievement) Indicator 12 : Number of projects supported by UKF with young scientists* Value (quantitative 0 15* 60 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments *new indicator and target added during 2009 restructuring. (400% achievement) (incl. % achievement) Number of projects supported by UKF successfully completed (as evaluated by the Indicator 13 : UKF evaluation committee)* Value (quantitative 0 30* 59 or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments *new indicator and target added during 2009 restructuring. (197% achievement) (incl. % achievement) Indicator 14 : Number of projects supported by UKF with Croatians living abroad* Value 0 30 37 (quantitative x or Qualitative) Date achieved 12/31/2006 05/31/2011 05/31/2011 Comments *new indicator and target added during 2009 restructuring. (123% achievement) (incl. % achievement) G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/20/2005 Satisfactory Satisfactory 0.00 2 06/14/2006 Satisfactory Satisfactory 0.10 3 02/23/2007 Moderately Satisfactory Moderately Satisfactory 1.89 4 09/17/2007 Moderately Satisfactory Moderately Satisfactory 2.63 5 02/14/2008 Moderately Satisfactory Moderately Satisfactory 5.86 6 06/13/2008 Moderately Satisfactory Moderately Satisfactory 8.56 7 06/27/2008 Moderately Satisfactory Moderately Satisfactory 10.11 8 06/30/2008 Moderately Satisfactory Moderately Satisfactory 10.70 9 01/30/2009 Moderately Satisfactory Satisfactory 14.04 10 08/15/2009 Satisfactory Satisfactory 17.90 11 04/30/2010 Satisfactory Satisfactory 28.47 12 12/20/2010 Satisfactory Satisfactory 33.33 13 06/13/2011 Satisfactory Satisfactory 39.79 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved Date(s) Restructuring Changes Made PDO Change DO IP in USD millions Reallocation of proceeds approved by CD:the Amendment Letter introduced two changes: (i) the ventures supported by RI for the purposes of commercializing (“Beneficiary Ventures�) became eligible for financing; and (ii) financing of the specific projects developed by the “beneficiary ventures� for the commercialization of intellectual 11/29/2007 MS MS 3.36 property innovation (licensing of intellectual property and the establishment of spin-offs) was allowed. EUR 2.6 million were reallocated to a new expenditure category (“Innovation financing�) and earmarked to RI. These changes were not considered as requiring project restructuring and were approved as reallocation of loan proceeds. xi Key changes: (a) revised PDO and associated outcome indicators, removing the objective of restructuring research and development institutions and therefore focusing on the objectives of enabling commercialization of public research and fostering private R&D; (b) the extension of the closing date and the 07/28/2009 Y MS S 17.90 reallocation of loan funds between disbursement categories, cancelling the subcomponent related to the creation of a venture capital fund(VENCRO) and increasing the allocation to programs that directly and indirectly support business R&D (RAZUM program and UKF component) Level 2 restructuring: reallocation of 12/07/2010 S S 32.05 proceeds Level 2 restructuring: reallocation of 03/31/2011 S S 36.82 proceeds Level 2 restructuring: reallocation of 05/15/2011 S S 36.90 proceeds If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Satisfactory Against Formally Revised PDO/Targets Satisfactory Overall (weighted) rating Satisfactory I. Disbursement Profile xii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Croatia‟s scientific and innovative performance deteriorated significantly during the 1990s. Public investments in R&D had substantially declined; a large number of scientists had moved abroad; research infrastructure had deteriorated and the country‟s overall scientific performance had worsened. Business R&D and the link between business and research organizations almost disappeared with the privatization of large state owned enterprises in the manufacturing sector. More broadly, Croatia lacked a coherent science and technology (S&T) policy and the appropriate funding mechanisms to implement it. In 2003, the Government submitted its application for EU membership and decided to prioritize R&D and innovation as sources of competitiveness in order to fully benefit from economic integration with the EU. 1 By 2004, Croatia indicated that it can accept the EU acquis on R&D and committed to adoption and implementation by accession.2 The Science and Technology Project (STP) aimed to help Croatia establish and develop the institutions and programs inherent to modern R&D and innovation systems. The rationale for Bank involvement was well grounded in the Country Assistance Strategy (CAS) program for FY05-09, which aimed to support the Government‟s growth and reform strategy for successful EU accession. STP was therefore expected to support Croatia‟s innovation-driven growth and competitiveness objectives.3 The Bank had been involved in a number of similar projects in the ECA region, including Russia and Turkey, and in other middle-income countries such as Brazil. The Bank‟s Operations Evaluation Department reviews concurred that the Bank had an important role to play in the area of S&T given its previous experience in the field. 1.2 Original Project Development Objectives (PDO) and Key Indicators The original PDO was two-fold: (A) strengthening and restructuring selected R&D institutions to promote applied research, while maintaining their scientific excellence; and (B) increasing the ability of enterprises to develop, use, adapt and commercialize technology. Key outcome indicators were: (i) Improved orientation of R&D towards industry needs, measured by number of joint projects of research and development institutes (RDIs) with industry; (ii) Increase in the self-sufficiency ratio of selected RDIs, measured by revenues earned from contractual research and services; (iii) Increase in the number of R&D projects commercialized (refined during implementation to measure the number of projects, not the percent); 1 Several important strategic documents were developed in order to drive Croatia towards this direction, such as the “Croatian Program for Innovative Technological Development� (2001), “Croatia in the Twenty-First Century-Science� (2003) and “Science and Education Act� (2003). 2 Due to its specificity, the EU acquis chapter on R&D did not require any transposition in national legislation, but rather existence of appropriate R&D policy, capable institutions, and conditions for effective participation in EU Framework Programs. 3 The contribution of the project to CAS objectives is explicitly presented in Chapter III of the CAS FY 05-09, in Table 5 on CAS and EU Accession Agenda Synergies, as well as in Annex B3a and Annex B3b. 1 (iv) Improved access to financing for R&D activities measured by the number and value of such financing provided; (v) Increase in firm-level productivity, market share, profitability, new products to market, and exports among beneficiary firms; and (vi) Increased collaboration between Croatian industry and research community with Croatian scientific Diaspora measured by number of joint ventures and applied research projects. 1.3 Revised PDO (as approved by original approving authority), Key Indicators, and reasons/justification4 The revised development objectives were to: (A) enable RDIs to commercialize research outputs; and (B) increase the ability of enterprises, particularly SMEs, to invest in R&D activities. Revised PDO, key indicators and targets aimed to reflect more accurately what could be achieved with the funds available, given the prevailing institutional conditions in the country, namely, limited appetite for RDI restructuring (as defined by the self-sufficiency indicator targets) and a commitment to increased research capacity and commercialization results of publicly-funded research that surpassed original expectations. In addition, the revised PDO aimed to refine the objective of upgrading the technological capacity of SMEs by focusing on the goal of increasing their private R&D capabilities (economic literature shows a close relationship between firm investments in R&D and innovation and productivity growth). As a result:  The corresponding key indicators (i) to (iii) described in the previous section were revised to reflect more relevant measures, key indicator (ii) was dropped since the objective of restructuring RDIs was removed, and key indicator (iv) was moved to a component indicator.  The corresponding original key indicators (v) and (vi) were dropped and (vi) became a component indicator; two new key indicators were introduced to reflect the revised PDO (b), namely the percentage of SMEs that introduced new and improved products to the market and the volume of private R&D mobilized through the programs supported by the project. 1.4 Main Beneficiaries Primary beneficiaries of the project were the Ministry of Science, Education and Sports (MSES); public RDIs – namely Brodarski Institute Ltd 5 (BI) and Rudjer Boskovic Institute (RBI); University of Zagreb, the largest in the country, and University Rijeka (both joining after the start of the project). MSES benefited from development and implementation of innovation programs by the Business Innovation Centre of Croatia (BICRO) – an agency focused on promoting R&D in SMEs and fostering emergence of technology based start-ups – and from the design and implementation of the Unity through Knowledge Fund (UKF) – a program supporting cooperation between local researchers and Croatian researchers living abroad. Knowledge-based SMEs and startups were direct project beneficiaries of programs run by BICRO. UKF programs mainly targeted research groups from national and foreign R&D institutions, while encouraging collaboration with the business sector through research contracts. By including RBI and University of Zagreb, the project covered approximately 75 percent of research in the country. 4 Approved on July 31, 2009; A summary of the PDOs and Project Outcomes Indicators is presented in Annex 2. 5 Originally a naval research institute providing services to the shipbuilding industry and the Navy of former Yugoslavia 2 1.5 Original Components (as approved)6 Component A: RDIs (EUR 11.5 million, of which EUR 10.5 million from the Loan). This component aimed at assisting with restructuring of RDIs. Activity areas included governance and strategic planning, human resources policies, business processes and information management, upgrade of physical infrastructure, as well as commercialization and intellectual property management. Subcomponents for at least two participating RDIs (BI and RBI) were envisaged. Component B: BICRO (EUR 17.1 million, of which from the Loan EUR 14 million). The component aimed to upgrade firms‟ technological capabilities by providing financing for development, use and adaptation of new technologies, and improving quality of products, ultimately contributing to the competitiveness of Croatian firms. Component C: UKF (EUR 5 million of which EUR 3.7 million from the Loan). This component was developed to attract Croatian scientists and researchers living abroad to nurture, cooperate and contribute to the development of R&D activities in Croatia. Component D: Project Implementation and National Innovation Support Group (EUR 1.8 million, of which EUR 1.64 million from the Loan). 1.6 Revised Components The component objectives, activities and results indicators were revised as part of the restructuring approved by the Board on July 9, 2009, as follows: Component A (revised): Encouraging commercialization of publicly-funded research. The objective was revised to foster commercialization of publicly funded research in place of implementing RDI restructuring. Two other RDIs joined the project in 2009: the Technology Transfer Office (TTO) of University of Zagreb and the TTO of University of Rijeka, with the objective of commercializing parent universities‟ research results. Component B (revised): R&D financing program for enterprises. This component was revised to increase the ability of SMEs to invest in R&D activities, by implementing publicly supported innovation-financing programs, with competent management and a transparent application process. The result framework by project components was also revised as part of the July 2009 restructuring (details presented in Annex 2). 1.7 Other significant changes Other changes resulting from the revised objectives and activities included funding reallocations; cancellations of loan funds; and extension of the closing date. These changes were formally approved at different stages in the period 2007-2011 (details on amendment documents included in Annex 2). Key changes included: (i) savings from cancelling sub-component B (iii) (creation of a venture capital fund) were reallocated to BICRO‟s programs – RAZUM and SPREAD (or 6 Detailed list of sub-components is provided in Annex 2. 3 IRCRO) – and to UKF;7 (ii) the loan amount was reduced from EUR 31 million to EUR 30 million, by cancelling EUR 1 million from the venture capital category to support BI‟s restructuring; and (iii) the project closing date was extended from November 30, 2009 to May 31, 2011 to allow full implementation of the envisaged project activities. Additional changes were approved in the July 2009 project restructuring: a) cancellation of BI‟s implementation covenant for maintaining an Institutional Renewal Account (IRA); 8 and (b) cancellation of BI‟s sustainability clause (project agreement with BI, Article IV, Section 4.02a). Amendments during project implementation and measures adopted to simplify Bank‟s sub- financing approvals and procedures were also integrated to the project‟s legal documents. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Project background analysis was sound in most aspects and was based on lessons from international best practices.9 Envisaging a full range of interventions across the entire spectrum of the Croatian innovation chain – including policy and institutional reforms, capacity building, and innovation-financing programs management – STP was conceived as the backbone of the Science and Technology Policy. In this regard, the ambitious and integrated project design was appropriate for the many needs of the S&T sector in Croatia, and its complexity was to some extent inevitable. The overall institutional and implementation arrangements, and the procurement and financial management capacity, as envisaged in the Project Appraisal Document (PAD), was appropriate to the complexity of the project design. The project had been correctly identified as a Substantial-Risk operation, with a number of risks identified in the PAD. Some of them, originally identified as substantial or moderate, turned out to cause major difficulties during implementation, while some of the proposed mitigation measures proved to be futile (details provided in section 2.2 and section 5). Aspects related to assessment of institutional quality at entry were too optimistic, however, affecting project implementation. First, institutional and capacity building requirements inherent to the project seem to have been underestimated. The adoption of a new Science and Technology Policy, for instance, delayed the effectiveness of the project for about twelve months – as stakeholder consultations proved longer than expected. Also, the level of capacity building required of beneficiary agencies – BICRO, UKF and RI – was greater than originally conceived. BICRO counted with a team of about five persons and a single program in the beginning – as compared to about thirty people, at least four programs and a portfolio of beneficiaries that increased tenfold at the end of the project. Fine-tuning the program manuals, having them 7 A revised result for the UKF component (operational on a sustainable basis), was motivated by a highly satisfactory performance of this component, though the restructuring did not envisage particular activities supporting this goal. With respect to the introduced result for the National Innovation System (consolidation), it was meant to acknowledge in retrospect the technical assistance provided to NIS over the life of the project. 8 BI was required to set up an internal revolving fund account and to make quarterly replenishments from its revenues to the account totaling EUR 1.5 million by 2010 (not done) and to use the amounts on deposit in said institutional renewal account for the purpose of financing its institutional renewal requirements, under terms and conditions acceptable to the Bank. 9 Most studies were funded by TAL 2 (Croatia Second Technical Assistance Loan). Annex 2 lists inputs for project preparation. 4 approved by government bodies, training personnel and developing the pipeline of applicants took longer than originally expected. As a result, implementation of sub-financing components – the BICRO and UKF programs – was delayed. There is no evidence that those circumstances have affected project outcomes. Rather, as they started disbursing heavily after the initial years, the programs received funds reallocated from underperforming components and ended the project with larger than originally planned investment levels. A factor affecting project implementation and outcomes is related to conditions to restructure RDIs in Croatia. This was evident when RBI, the largest research institute in the country and the second to be restructured, joined the project several months after its effectiveness and under a limited goal of promoting commercialization. The limited support for restructuring BI became evident when key restructuring milestones – related to cost reductions and revisions of human resource and management policies – were systematically missed during the first years of implementation. This, in turn, contrasted with a satisfactory performance in terms of diversifying research areas (away from shipbuilding) and clients (towards private sector and international research funds) for which project support was instrumental. 10 It did not help that the Government‟s obligation to provide state budget support to BI to finance costs associated with its Hydrodynamics Laboratory, an integral part of the restructuring program, and another project covenant never materialized.11 Those circumstances affected one of the original PDOs (and thus the outcome, namely restructuring RDIs). 2.2 Implementation As the following pages will demonstrate, the project was implemented successfully (the overall rating is satisfactory): despite the difficulties during the first phase of implementation, the project achieved its objectives, with good results. There was a range of factors contributing to this outcome, all of them indicating solid Government support to the project and to the broader R&D and innovation policy agenda in Croatia, within the context of completing the EU accession process. The global economic crisis forced the project to review one of its original components, namely development of a venture capital fund, but did not affect is overall results.12 Despite fiscal constraints imposed by the global economic crisis and the related uncertainties, the Government managed to provide sufficient fiscal space and counter-part funds for project implementation. 13 It adopted policies and proposed laws and regulations, when necessary, to support the PDO and specific project activities; improved policy framework conditions for innovation (as reflected by a lessened tax base for R&D spending and harmonization of the intellectual property (IP) regime with EU and international legislation). A new science law aimed 10 This deficiency was identified in mid-term and 2009 Joint Portfolio reviews. After using more than half of its original allocation, BI was lagging behind in achieving the self sustainability ratio needed for successful restructuring. Interestingly enough, BI access to direct subsidy had been close to null and most of its revenues were obtained through a competitive process or contract research. Yet, the institute had systematically showed slightly positive financial results. Meanwhile, RBI‟s interest in joining STP was de facto limited to improving management and commercialization. 11 Section 2.07 of the Subsidiary Finance Agreement 12 The issue was addressed in the 2009 restructuring. 13 The risk of insufficient counterpart funding required careful management of the Government‟s contribution, in order to avoid major delays in BICRO‟s activities. Once the agreement was reached with the State Treasury on budget allocation adjustments at the end of 2009, the issue of counterpart funding was resolved. 5 at improving conditions for management of public research was publicly debated. In addition, the Government appointed a capable and committed staff for the corresponding project agencies and the Project Management Unit (PMU). Downsides in performance were manifested by occasional fluctuations in attention to project issues, due to government reorganizations. At implementing agency level, MSES adopted streamlined procedures to delegate responsibility to agencies and reviewed project documents in the initial phase of the project. BICRO, for instance, introduced a new program, the Proof of Concept, to better fit the needs of small scale early-stage financing in the country – complementing the RAZUM and SPREAD programs originally planned by the project. UKF‟s team successfully tailored the programs initially conceived, adjusting them to local reality and to the need to foster integration to the European Research Area (ERA) through higher participation rates in EU programs such as the Seventh Framework Program (FP7), EU‟s main instrument for funding research. Both teams were actively pursuing the development of a solid pipeline of project proposals by SMEs and academic sectors and were fully committed to the establishment of transparent, meritocratic and relatively simple selection processes, contributing to an increased number of potential applicants. Monitoring and evaluation (M&E) schemes were established at inception of those programs, helping the management team to adjust their regulations based on timely feedback from applicants (both institutions commissioned an independent assessment of their programs). The global economic and financial crisis eliminated the chances of developing a local venture capital fund. After several attempts to identify private investors willing to match the funds made available by the project, it became clear that the global deleveraging of financial markets had affected Croatia significantly with less liquidity available in the market and more risk-averse investors. It also became clear that international investors were not willing to invest in the country as venture funds, also affected by the global crisis, were concentrating resources in their previous investments rather than considering options for expanding their portfolios. 2. 3 Monitoring and Evaluation Design, Implementation and Utilization Institutional arrangements for M&E were put in place during project preparation and were appropriately carried out during implementation. The original results framework had clear objectives, with all but one of the outcome indicators being realistically set. The results framework had coherent linkages between outputs and outcomes, measurable indicators and clear data collecting methods. Moreover, the results framework was incorporated in relevant project documents, as required. MSES/PMU regularly provided reports on project implementation, including updated information on outcome and output indicators. Regular monitoring based on the M&E framework and risk assessment allowed the Government and the Bank to take informed decisions during implementation, including on project restructuring and reallocation of funds to better performing activities. The Government also used the M&E framework for internal accountability purposes, such as reporting on progress made in implementing the Science and Technology Policy as part of EU accession talks, and having better insight of R&D programs and policy making. Most of the indicators were formally introduced by the Government in the “Action Plan for increasing the level of investment in research and development.� Finally, a first assessment of the BICRO and UKF programs and the 6 existing tax-breaks for R&D activities was commissioned by the Government to the Zagreb Institute of Economics. 2.4 Safeguard and Fiduciary Compliance FM arrangements were rated Satisfactory during the project‟s life, with supervision missions routinely monitoring fiduciary compliance and with annual audits performed by external auditors. On one occasion, the rating was downgraded to Moderately Satisfactory due to delays in submitting 2006 audit reports and poor result of BICRO‟s, BI‟s and RBI‟s financial statements. The corrective measures recommended by the Bank were implemented in a timely fashion. Although counterpart funding was mostly rated as moderately satisfactory, implementation was not constrained, as the Government was taking corrective measures during the State Budget rectification. In 2009, given the country‟s continuous fiscal constraints and following the request of the Government, the Bank increased financing from the loan proceeds for the RAZUM program, from EUR 3 million at appraisal to EUR 7.5 million as the final amount. In regards to procurement, the PMU procurement staff benefitted from support of the Bank‟s procurement specialist based in the field office, enhancing their knowledge of Bank procedures and increasing overall capacity to support beneficiaries. 2.5 Post-completion Operation/Next Phase The lack of sustainability of the programs developed through STP has been and remains a major issue for the project. The public innovation policies and programs developed under the project, and the improved interaction between companies and research institutes in S&T are jeopardized by the fiscal limitations and lack of flexibility of MSES to reallocate funds internally. In the period 2014-2020, Croatia will have an annual amount of approximately EUR 150 million available for R&D from EU Structural and Cohesion Funds, available upon EU accession. Access to EU funds after 2013 could help mitigate the constraints to a sustained increase in public R&D expenditures imposed by the country‟s fiscal consolidation needs. In this context, the Government requested the Bank to prepare a follow-up project to STP, with the primary objective to ensure a high absorption rate of EU funds, by strengthening the policy and institutional frameworks and further consolidating the institutional project management capacity. At the same time, to address the funding gap, the project would support firm-level R&D for the period ahead of EU accession and prepare a pipeline of project proposals for the use of EU funds. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The project development objective remains highly relevant for Croatia‟s economic development and is central to the Europe 2020 Strategy‟s Innovation Union flagship – one of the seven flagships aimed at improving conditions and access to finance for R&D and innovation. A recent 7 calculation estimates the social rate of return on R&D investments in Croatia to 73 percent – more than twice, for instance, the rate of return for infrastructure.14 Yet Croatia‟s aggregate R&D investments are at a relatively low level (0.9 percent of GDP in 2011, as compared to 3-5 percent in innovation-driven economies such as Finland, Sweden and Israel). R&D and innovation continues to hold the key to boosting productivity and securing long term development. Project objectives are aligned with the objectives of the current CPS for FY09-FY12, particularly with its second pillar of strengthened private sector led growth and accelerated EU convergence. In response to country needs in view of EU accession, these objectives have been further calibrated by the CPS Progress Report to gain more focus towards EU 2020 strategic objectives and accelerate absorption of EU funds. 15 More broadly, the PDO is aligned with EU‟s development priorities as articulated in the Europe 2020 Strategy and its “smart growth� pillar – aimed at boosting EU‟s performance in R&D and innovation. Project design was appropriate for achieving the development objectives: (i) a mix of instruments to alleviate market failures and boost innovation was envisaged; (ii) different stages of the innovation process (from basic and applied research to product development and commercialization) were targeted; and (iii) partnership between business and academia and collaboration between local scientists and Croatian Diaspora were encouraged. Meanwhile, the national innovation policy supported by the project ensured coherence and integration of the different STP components. The flexibility built into the project design proved to be beneficial in balancing the different components and to respond to country needs. Through restructurings, project funds were directed to project activities with most additionality benefits. Also, in response to the fiscal constraints faced by the country, the financing ratio from the loan was increased, particularly in the case of sub-financing categories (RAZUM). 3.2 Achievement of Project Development Objectives The project enabled R&D institutions to commercialize research outputs. It also increased the ability of enterprises to develop, use and adapt technology by addressing and correcting the market failures of programs focusing on R&D financing for enterprises (RAZUM and SPREAD). The project also contributed to the higher-level objectives of increasing competiveness of Croatian firms and integration with the EU. Competitiveness derives from the creation of differentiated capabilities, created through innovation and needed to sustain growth. 16 The project outcome indicators were met or exceeded in all but two cases (the number of concluded spillovers and the value of licensing agreements where the performance was slightly lower than the estimated target). Key achievements included: (i) more research outputs and research capacity commercialized by RDIs; (ii) improved scientific and technological cooperation; and (iii) more firms investing in R&D activities, as illustrated below by the following results:17 14 Seker, Murat (2011). Estimating the impact of R&D and Innovation in Croatia. Background paper prepared for the "Croatia Policy Notes for Incoming Government". World Bank, 2012 (EW-P127664-ESW-BB) 15 CPS FY09-FY12 Progress Report, August 2011; Report Number: 61715-HR 16 See a review of the literature by Clark, John and Ken Guy (1998), “Innovation and Competitiveness: a review�, Technology Analysis and Strategic Management, 10(3). 17 A more detailed explanation of the project achievements for all of its four components are presented in Annex 2. 8  69 new research contracts (target was 55) with industry were concluded by BI and UKF, with a total value of EUR 10.9 million (against the target of EUR 9.8 million).  Rudjer Innovations (RI) (RBI‟s TTO) established five spillover companies (two new companies to be completed within the next 12-24 months). Total value of the five spillovers amounts to EUR 3.1 million (target was EUR 2.9 million for eight spillovers).  RI signed 12 licensing agreements (exceeding the target of eight agreements), including with the Massachusetts Institute of Technology and Brown University, reaching about EUR 800,000 (falling short of the target EUR 1.0 million).  UKF‟s cooperation program established collaboration with 133 foreign research institutions including Mack-Planck Institute, Swiss Federal Institute of Technology, and Johns Hopkins University.  Increased volume of private funding for R&D activities mobilized by firms: EUR 13.7 million, 30 percent higher than the initially envisaged target of EUR 9.2 million.18 3.3 Efficiency The project is assessed to have achieved a high level of efficiency. The PAD justified the efficiency of STP based on (i) results from developed countries showing that the rate of return on R&D investments is around 20 percent, and (ii) the public good nature of the intervention (but not automatically to the project). Cost-benefit analyses are however not easily applicable to evaluating efficiency of the project for a number of reasons, most notably due to the positive externalities related to R&D and the gap between social (economic) and private rates of return – the latter measured by internal rates of return (IRR). IRR in the present case would also be highly dependent on the values assigned to the results of commercialization activities and to the intangible benefits related to capacity building. In the absence of standard cost-benefit analysis, and following ICR Guidelines, project efficiency is evaluated through estimates of (i) social opportunity costs and (ii) cost effectiveness. To estimate social opportunity cost of R&D investments, we compare the rate of return on R&D with other possible public investments in Croatia.19 Our findings show that the estimated rate of return on R&D (73 percent) is at least twice the value for infrastructure and seven times higher than for education. To assess the project‟s cost effectiveness (i.e. whether it represents the least- cost solution to attain identified results), we consider whether the development objectives would have been achieved without STP support (see Annex 3). 20  Research Commercialization. It is unlikely that the observed volume of commercialization would have been achieved without the technical assistance and financing provided through STP to the TTOs and BI. SPREAD, in particular, subsidized collaboration between the public 18 This amount reflects only the direct contribution of firms participating in the project programs (at least 30% of RAZUM projects; at least 50% of SPREAD projects; 20% of UKF contracts). 19 For practical reasons, we chose to limit the comparison to investments in education and infrastructure – the bulk of the projects of the country‟s 2010 Economic Recovery Program (see Annex 3 for details). 20 We consider the initiative to be a least-cost option if the results cannot be achieved otherwise. 9 and private sector. In a survey prepared by the Zagreb Institute of Economics (ZIE) SPREAD beneficiaries agreed on average that collaboration with academia would not exist without funding. Firms also believe that without such collaboration, it would have taken longer to achieve the same results, while the research scope would be significantly reduced. 21  Business R&D spending. The ZIE survey also indicates that a large share of RAZUM- mobilized business R&D spending would not have occurred in the absence of the program. Smaller budgets would have resulted in longer duration and smaller scope of the activity, lower R&D capacity, and a lower innovativeness level. 22 Data also showed that R&D intensity increased after start of the RAZUM program in 9 out of 15 companies. RAZUM also generated new employment (54 new jobs), most of which has been in R&D (48 jobs). The companies expressed their intention to keep the new staff after the project is over, while more than half of the companies would have been unable to hire new staff for the project.  Research Excellence. A similar result was achieved for UKF, where the survey indicated that absence of funding would have resulted in lower quality of scientific output; fewer papers published in top ranked journals; and more difficulties for researchers to cooperate with Croatian Diaspora and foreign scientists, as well as to educate young researchers abroad. Another important result that helps to understand the impact of UKF in terms of research excellence is that projects supported by UKF were twice as likely to obtain additional funding from FP7 as the average Croatian project application. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory The overall outcome of the project is Satisfactory based on a moderately satisfactory outcome before the 2009 restructuring and a satisfactory outcome after restructuring.23 If assessed against the original development objectives the project outcomes would have been rated moderately satisfactory.24 However, the project restructuring that included the revision of the original PDOs was done at an early stage with 37 percent of the loan funds disbursed, leaving enough space for achievement of the new objectives. By project closing date, all but two of the project outcome indicators (the number of spin-off companies and the value of licensing agreements with the business sector) were met. The project objectives, design and implementation remained highly relevant for the current country context. 21 Before the project, the mindset of researchers was unfavorable to the concept of commercialization resources (financing and knowledge) were not available. In the project period, there were no other similar activities being implemented in Croatia. 22 While the “additionality effect� of public support to business R&D has been a topic of debate for the last decades, the more rigorous evaluations of recent years have consistently found slightly positive (but significant) effects of public support to R&D (see Andres, Luis, Borja C. and Correa P. (2012) “R&D Interventions: Where do we stand? A meta-analysis of impact evaluation studies,� mimeo) 23 Rating is weighted in proportion to the share of disbursement before and after restructuring, according to the "Joint Guidelines for ICRs and IEG assessments" (Appendix B of ICR Guidelines). 24 (i) Although the self sufficiency ratio of BI reached 95 percent by end project (instead of targeted 100 percent), this outcome is still volatile as the institute needs further reductions of costs and a more stable revenue base. (ii) RBI did not join the project as restructuring candidate and limited its participation to improved commercialization. 10 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development NA (b) Institutional Strengthening The project had a substantial impact on institutional development across the science and innovation system in Croatia, as indicated by such achievements as: (i) UKF selected as “Best Practice� by the European Regional Economic Forum (EREF) on the topic �Developing human capital and managing migration for more competitive European regions;� (ii) UKF selected as “Good Practice� by the International Labor Organization (ILO) on the topic “Promoting linkages between migration and development;� (iii) BI and RI becoming members of EARTO and participated in FP8 Task Force meetings; (iv) BI strengthening cooperation with the innovation network PERA/iNET for preparation of projects to be financed from FP7; and (v) BICRO being awarded “Development of Innovative Business Parks to Foster Innovation and Entrepreneurship in the SEE Area,� and becoming member of the European Enterprise Network and the European Business and Innovation Network. (c) Other Unintended Outcomes and Impacts (positive or negative) The project components were aligned with the Europe 2020 Innovation Union initiative and contributed to the dialogue related to Croatia‟s EU integration. The average innovation performance (AIP) indicator measures countries‟ progress towards the EU2020 Innovation Union.25 Incremental changes of this indicator that could be attributable to the project cannot be quantified at this stage given insufficient comparable data. However, a qualitative assessment could be safely done as the project outcomes/outputs and the additionality benefits have an impact on 20 of the 24 sub-indicators that compose the AIP (Table A.3.3 presented in Annex 3). Overall, by meeting/exceeding its objectives, the project had impact on establishing and improving the channels that lead to a more knowledge-based society. Through the UKF program, the project supported participation in the EU Framework Programs and ERA objectives. FP7 funds awarded to UKF projects represent 34 percent of all FP7 funds attracted by Croatian scientists in 2007-2010, demonstrating high competitiveness of UKF- supported research groups. In addition, UKF and MSES were invited in December 2011 to negotiate a COFUND project within FP7 for fellowship, training and career development. The project would help integrate Croatian scientists in ERA and foster incoming mobility to Croatia. As part of the UKF “Across the Borders Program,� the project resulted in publication of 54 academic papers in journals and 73 papers in conference proceedings. Beneficiaries also submitted 30 applications to the FP7 European research program and 19 applications to other international research programs. They established long-term cooperation with Croatian Diaspora scientists and with scientific/research institutions from abroad. 25 The average performance is measured using a composite indicator building on data for 24 indicators going from a lowest possible performance of 0 to a maximum possible performance of 1. 11 4. Assessment of Risk to Development Outcome Rating: Substantial The risk to the development outcome is rated as Substantial given the probability of occurrence and likely threats to the described outcomes. A first risk relates to the sustainability of key institutions/programs since elections in December 2011. A second risk is the availability of innovation financing mechanisms. As a mitigating factor, the Government has started to prepare with the Bank a Second Science and Technology Project focused on preparing Croatia‟s R&D and innovation sector to efficiently absorb EU funds expected to be available from July 2013. The new Government is currently considering a merger of UKF with the National Science Foundation – an option that could seriously affect the transparency and merit-based approach to selection of scientific projects to be funded and thus the excellence of its research results, given NSF‟s constraining regulations and governance structure. BICRO‟s transition from a private liability company to a government agency, initiated by the previous administration and not yet completed, is likely to have at least two negative implications: (i) salary cuts of up to 40 percent, which create a severe risk of losing the cadre of highly trained professionals; and (ii) and a more restrained capacity to manage multiyear budgets, which increases BICRO‟s dependence to the budget cycle, contributing to the volatility of public financing for business R&D. The availability of budgetary resources to existing BICRO early-stage financing programs remains uncertain in the context of currently severe fiscal constraints, even though the MSES 2012 budget for R&D increased relative to 2011.26 Moreover, markets are unlikely to provide angel/venture financing to the pipeline of innovation projects and technology-based startups that were generated by STP – a rare pipeline of potentially viable projects as evaluated by international experts (see the STP Aide Memoire of February 14-21, 2011). Developing a venture capital industry remains a mid-term challenge for Croatia and resources of the Economic Cooperation Funds program to support the equity industry are likely to trickle down, benefiting technology based projects. The risk related to the existing funding gap has been recognized by the European Investment Fund in the study “Western Balkans Venture Capital Market Assessment� (Nov. 2010).27 The Second Science and Technology Project (STP II), currently under preparation as a follow-up operation, aims to further strengthen Croatia‟s R&D sector in order to maximize absorption of future EU funds, estimated at about EUR 150 million annually (2014-2020) – an amount sufficient to efficiently address Croatia innovation financing needs in the near future. Among other initiatives, STP II will assist the Government in adjusting current programs (BICRO and UKF) to EU regulations and preparing submissions to EU Structural and Cohesion funds. In 26 Planned MSES budget for R&D is KUN 687.5 million in 2012 (7.7% percent increase from 2012). Excluding technology, this ratio, relative to 2011, is even higher (+10.6%). When adding planned spending on public institutes, the corresponding value is KUN 968.2 million in 2012 (6.42 % increase from 2011). 27 In the referred report, EIF conclude that: “(…) In Croatia, TT [technology transfer] financing is significantly developed because of the World Bank sponsored Science and Technology Project (STP). Yet, TT financing is in pioneering stage with potential to increase from current EUR 5 million per annum to EUR 20 million per annum in 2013-2014. The most serious lack of TT funding is between STP end-date (May 2011) and the EU Accession. Current estimate of unsatisfied expressed demand is between EUR 6-9 million, unexpressed demand is larger.� (p.7) 12 parallel, the Government and BICRO, under leadership of Croatia‟s Development Bank (HBOR) are forcefully pursuing strategies to foster the creation of sound venture financing instruments. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory The project was prepared with extensive consultations and stakeholder discussions and with significant funding support from TAL 2. The project design was complex and was grounded on both theoretical concepts derived from the relevant economic literature and on lessons from international best practices. The project responded well to the Government‟s innovation led growth strategy and to EU accession goals. Significant efforts were made to ensure effective implementation arrangements, given the number of agencies involved and diversity of instruments utilized. Important project documents and frameworks were developed and efforts were made to strengthen the institutional capacity. The Operational Manual, business plans of participating agencies, M&E frameworks, financial management and procurement arrangements were thus developed and training courses for procurement delivered. Although project readiness was achieved to a large extent, as it turned out during the first years of implementation, incomplete project readiness and weak design caused delays and affected timely achievement of PDOs. Despite the high risks and uncertainties identified at project appraisal and negotiations, project objectives were not narrowed down, nor were the problematic aspects of the project design addressed. Thus, some project design issues which were not entirely resolved at preparation had to be addressed during implementation. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Satisfactory The Bank provided consistent and timely implementation support with regular missions and a comprehensive mid-term review (MTR) mission in 2008. The Implementation Status Reports (ISRs) were realistic and informative and addressed promptly the project‟s problems. The Aide Memoires were regularly prepared and focused the Government and Project agencies‟ attention to resolving project execution issues by suggesting remedies in a timely manner. Besides the regular supervision mission, close contacts with project agencies were kept through frequent communications and interactions with the local project officer and the decentralized procurement and financial management specialists. The appointment in 2007 of a new TTL and a local project officer in Zagreb gave new impetus to project implementation and reinforced the focus on PDOs. Actions were taken by the new project team to streamline project implementation and decentralize some of the project decisions at local level. The MTR constituted the cornerstone for the satisfactory turnaround of project implementation, by proactively addressing the structural weaknesses of the project. The opportunistic project restructurings approved in 2010 and 2011 directed project funds towards 13 activities with most additionality benefits. The Joint Portfolio Reviews of the Bank and the Government were also instrumental in proactively signaling the project issues, particularly during the first years of project implementation. Moreover, the Bank team provided technical assistance to national policy issues in S&T, among others through input to the Science and Technology Policy and the draft Science Act. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The overall Bank performance was moderately satisfactory due to shortcomings in project preparation and appraisal, in spite of the satisfactory Bank performance during implementation and the satisfactory outcome of the project. The flaws in design and weaknesses in project preparation hindered smooth project implementation and jeopardized project outcomes. The sustained efforts made during implementation led to partial recovery of the initial delays in project implementation, while the proactive project restructuring allowed for satisfactory achievement of the project outcomes. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory Performance of the Government/MSES is rated moderately satisfactory, mainly due to weak ownership during the first years of implementation. Commitment to S&T goals was reflected by the adoption of strategic research policy documents, primarily the “Science and Technology Policy 2006-2010,� the “Action Plan 2007-2010� and the “Action Plan for increasing the level of investment in research and development,� but public consultations on these documents did delay project effectiveness by twelve months. Other delays in the initial phases were related to fluctuations in attention to project issues due to Government reorganizations and bureaucratic red tape. Moreover, the risk of insufficient counterpart funding required careful management of the Government‟s contribution in order to avoid major delays in BICRO programs. This risk was addressed through an increase of the loan contribution for RAZUM and agreement with the state treasury on adjustment of budget allocations to project needs. In later project stages, MSES took the necessary actions for resolution of project implementation issues, including: (i) adequate staffing throughout the project, with a well-performing PMU; (ii) adoption of streamlined procedures to delegate project implementation responsibilities to project agencies; (iii) timely revision/adoption of project documents, thus departing from the slow performance in the initial phase; (iv) engagement in project coordination at state secretary level; (v) readiness to foster commercialization by allocating funds to the TTOs of Universities of Zagreb and Rijeka; and (vi) active participation in Joint Portfolio Reviews with the Ministry of Finance. Meanwhile, MSES commitment to the revised project objectives led to timely execution of the revised project schedule and disbursement of 60 percent of loan funds within 18 months. 14 (b) Implementing Agencies Performance Rating: Satisfactory BICRO and BI were ready for implementation as they benefitted from TAL 2 financing for training, equipment, and expert assistance. Once MSES had addressed the necessary legal arrangements, the agencies engaged with full intensity and remained highly committed throughout the project. The BICRO component has been rated moderately satisfactory for a certain time given the slow start of its SPREAD program. This was addressed at mid-term with adjustments that led to a high interest from the business sector and an increase in the program‟s allocation. BICRO‟s proactive approach to fostering private R&D activities was demonstrated by their development of the Proof of Concept program (within TechCro), which was a missing link in the innovation financing chain. UKF performance was rated highly satisfactory throughout the project. The Bank delegated management of funds to a small and highly efficient team of executives and representatives of academia and private sector. UKF‟s good performance resulted in international recognition of their practices by the ILO. The RBI subcomponent was delayed due to the need to adjust project financing to go through RI (a newly established TTO) as well as the time needed to identify a team with unique skills to foster technology transfer, which were not readily available on the labor market. Upon setting the full team at RI, the activities picked up and RI had its first spill-over companies established in line with annual project targets. Though the views of BI for the restructuring goals varied throughout the project, its management remained eager to continue the dialogue with the Bank until a set of interventions was agreed that helped BI increase its competitiveness. This resulted in increasing the innovation capacity of BI, as indicated by awards of UKF grants to BI research; awards of competitive funds from FP7 with foreign partners; and increased networking and membership in the international network of European R&D institutions. Coordination was established among the agencies, which efficiently developed a good practice of sharing data on beneficiaries, including a referencing system for particular R&D programs. The agencies also benefitted mutually from the respective programs – generating added value to the final beneficiaries, both among SMEs and research groups. For instance, BICRO‟s pipeline for SPREAD benefitted from companies that were identified by RI, while BI benefitted from BICRO and UKF programs. Universities of Zagreb and Rijeka shared documents and experiences and participated in UKF programs together with researchers from RBI and BI. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory The ICR rates the overall Borrower performance as moderately satisfactory. Government and MSES actions demonstrated their commitment to foster closer links between knowledge institutions and the business sector, with renewed ownership of MSES, as of mid-term review, with integration of the project within sector priorities. The agencies‟ commitment was equally demonstrated by their responsiveness to project obligations, and initiatives in adjusting and developing R&D programs that were conducive to generating innovation, and using the knowledge for economic benefits to the society. Nevertheless, there were still various bottlenecks encountered in the first two years of the project. These include a lack of progress on BI and RBI restructuring, as envisaged in the original PDO, which prompted project 15 restructuring; conditional loans imposed on SPREAD (as opposed to agreement on providing matching grants); and the issue of counterpart financing. While these have been resolved thanks to commitments of MSES and the agencies, they did contribute to implementation delays. 6. Lessons Learned A first lesson learned from the Science and Technology Project is that innovation related projects may achieve tangible results within a relative short time. STP achievements indicate that Croatia‟s innovation performance was not constrained by a lack of ideas, as sometimes argued, but by the availability of appropriate institutions – such as TTOs, early-stage financing and mentoring services. By the end of the project, BICRO‟s pipeline of projects was large, exceeding many times the amount of annually available funding. A change in mindset within the scientific community gradually unfolded, as initial results showed that academic excellence and research commercialization were not contradictory activities. A lesson potentially valid to several middle- income countries, therefore, is that untapped sources of innovation could be unleashed provided that an adequate institutional environment is in place. A second lesson derives from factors seemingly critical for STP achievement: (i) focus on well defined, quantifiable targets (enabling commercialization of publicly funded research and supporting SME investments in R&D); (ii) design of a comprehensive set of instruments targeting informational and financial bottlenecks inherent to innovation processes (TTOs and BICRO programs); (iii) a transparent and meritocratic governance regime of the support programs; and (iv) monitoring schemes put in place in the beginning, allowing the necessary adjustments to materialize. More specific lessons in terms of the instruments used are:  BICRO‟s pro-active role in nurturing and developing ideas into financeable projects was critical for the development of a sound pipeline – a result that countries without equivalent organizations often fail to achieve. This role, combined with provision of different financing instruments to different stages of the innovation chain, may represent the main advantage of organizations dedicated to promotion of knowledge-driven startups and R&D in SMEs.  The UKF Cooperability Program shows that it is possible to mobilize scientific Diaspora in order to significantly contribute to the excellence of local research. One reason for the program‟s success is the focus on promoting collaboration rather than “bringing back� scientists living abroad – as is often the case of efforts to reverse "brain drain.� A third lesson can be learned from what STP could not achieve, namely: restructuring research institutes and promoting a venture capital industry. The failure in fostering restructuring of research organizations was caused by a combination of political and technical difficulties. While a complex political economy is to some extent inherent to any restructuring process, a technical difficulty found in STP was related to definition of the own concept of restructuring, as BI did not count with direct government support and was financially self-sustained. Moreover, the appropriate combination of public and private funding will depend on the role of the research organization within the country‟s S&T policy and can hardly be defined ex-ante.28 The lesson 28 Unlike privatization of state-owned enterprises, where the goal is to transfer potentially viable firms and close down non- sustainable ones, there might be a rationale for supporting research organizations that are not commercially sustainable due to the 16 therefore seems to be that restructuring public research organization would be better undertaken as a result of the consensus built around their role in Croatia‟s S&T policy and the corresponding research infrastructure strategy. STP seems also to have underestimated the complexities in developing market-driven venture funds in the country. The request for the managing firm to mobilize additional EUR 10 million during the financial crisis was per se a challenging task, as indicated by the dramatic drop in venture capital financing worldwide. The lack of a clear portfolio of prospective investments at the time the program was implemented (half way through project implementation) imposed additional difficulties and did not contribute to attractiveness of the investment. In addition, Croatia‟s investors have shown to be unprepared to take (and manage) this type of risk of technological (rather than commercial) nature and with longer maturity periods (7-10 years). The lessons in this case seem related to the importance of a tangible portfolio of investment opportunities and of technical assistance (in the form of “on the job training�) to jump-start the private market for venture financing in middle-income economies. A fourth lesson relates to sustainability of achieved results. By the end of the project, funds for continuation of the newly created programs had not been secured. The legal status of BICRO and UKF is currently being reviewed by the new administration. TTOs still have to fully integrate with their parent organizations (RBI and Universities of Zagreb and Rijeka) and are currently struggling to finance operating costs. Solutions, however, are envisaged. The Government plans to use EU funds (available as of mid 2013) to expand and stabilize the funding gap. The Science and Innovation Investment Fund (an IPA funded program) has been established to partially finance TTO activities. A follow up operation to STP (STP II), focused on preparing the R&D sector to efficiently absorb EU funds has been requested from the Bank and is under preparation. Yet, facing severe fiscal constraints and a relatively rigid budget structure, the Government failed to prioritize the support for R&D programs – despite the explicit recognition of the importance of innovation for Croatia‟s economic development in the coming years. This in turn reflects challenges faced by transition economies in mainstreaming innovation policies. In order to support client countries in mainstreaming innovation policies, a programmatic approach may be needed – combining different Bank instruments (development policy loans; technical assistance and program for results loans) in order to sequentially address the different challenges in designing, managing and evaluating innovation policies. A full-fledged country strategy aimed at increasing the contribution of knowledge to development as part of the CPS could help establish a framework for Bank support. This would also allow for adjustment of policy instruments to evolving country conditions, thus increasing Bank support effectiveness. This approach cannot be achieved in one, but a series of projects. In the case of Croatia, the World Bank Second Technical Assistance Loan (TAL 2) tried de facto to provide for the key reforms and capacity building while the envisaged STP II would build on the achievement and lessons derived from STP and respond to new country challenges and opportunities related to full EU integration. positive externalities of noncommercial research. The problem looks like much more of a public-private partnership; in other words, how much public support is really needed for those organizations that lie between pure basic research (full support) and pure marketable research (zero support). 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners29 (a) Borrower/implementing agencies NA (b) Cofinanciers NA 29 See Annex 7 for the Borrower‟s ICR. 18 ANNEX Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual Component Appraisal Estimate Percentage of Appraisal Estimate Loan Total Cost Loan Total Costs % % A. Restructuring of RDIs 10.5 11.5 8.2 9.8 78% 86% 1. BI 4 4.7 3.0 3.6 75% 77% 2. RBI / RI 4 4.3 4.5 5.3 111% 123% 3. University of Zagreb 0.5 0.5 0.5 0.6 99% 121% 4. Other RDIs 2 2 0.3 0.3 14% 17% B. BICRO 14 17.1 15.8 17.8 113% 104% 1. Institutional building 0.7 0.9 0.6 0.8 93% 86% 2. RAZUM 3 5 7.5 8.0 250% 161% 3. Venture Capital 4.5 4.5 - - 0% 0% 4. Tech Centers 4.3 5 5.6 7.0 131% 141% 5. SPREAD 1.5 1.7 2.0 2.0 133% 118% C. UKF 3.7 5 4.5 5.7 122% 113% 1. Operational support 0.3 0.5 0.3 0.4 113% 88% 2. UKF fund 3.4 4.5 4.2 5.2 123% 116% D. Project impl. support 1.64 1.8 1.4 2.0 86% 111% 1.PMU 1.2 1.2 1.2 1.7 96% 138% 2.National Innovation Strategy 0.44 0.6 0.3 0.3 60% 55% Unallocated and fees 1 1.3 0.1 0.1 8% 6% Unallocated 1 1.14 - - 0% 0% Fees 0.16 0.16 0.1 0.1 48% 48% Total 31 36.7 30.0 35.4 97% 96% (b) Financing Estimate Actual/Latest Type of Percentage of Source of Funds at appraisal estimate Cofinancing appraisal (%) (EUR million) (EUR million) Borrower 5.7 5.4 94.7 International Bank for Reconstruction and 31 30 96.8 Development 19 Annex 2. Project Development Objective, Outputs and Results Framework by Component Original Development Objective30 Revised Development Objective Loan Agreement: (a) to strengthen and restructure research and development (a) to enable research and development institutions and to promote commercialization of the research outputs and (b) institutions to commercialize research outputs; to upgrade technological capabilities of enterprises. and (b) to increase the ability of enterprises, PAD main text: (a) to strengthen and restructure selected research & particularly small and medium sized companies, development institutions to promote applied research, while maintaining to invest in research and development (R&D) their scientific excellence; and (b) to increase the ability of enterprises to activities. develop, use, adapt and commercialize technology. PAD Annex 3: (a) to strengthen and restructure research and development institutions to promote applied research and increase their self-sufficiency and (b) to increase the ability of beneficiary enterprises to develop, use and adapt technology. Target Revised or new outcome Baseline Original outcome indicators Baseline Target 2009 Change indicators 2006 or (PAD, Annex 3) 2006 2011 (201031) (Project Paper) later 1. Improved orientation of 0 30 (50) Revised to 1. Increased number / value (€M) 2 / €0.5 30 / R&D better of research contracts with the €8.0 towards industry needs indicate industry (BI, UKF contracts co- measured by number of what is financed by business, and TTOs contracts with the industry measured contracts with business not (cumulative) financed by STP) 2. % increase in self- 75 100 Dropped sufficiency ratios of RDIs (100) since (BI) the objective to restructure RDIs was removed 3. % increase in number of 0 8 (15) Revised to 2. Increased commercialization of R&D projects better R&D results as signed by RI and commercialized (refined define what other TTOs : 0 8/ during implementation to is measured a) Number / value (€M) of €2.8 measure the number of spillovers (incl. joint ventures) 0 projects, not the percent) b) Number / value (€M) of 8/ licensing agreements signed with €1.0 business 4. Improved access to 3 4 (4) Dropped – financing for R&D activities and moved (the number of programs32) as component indicator 5. Improved productivity 0 >15% Dropped as Improved market share and 0 (25%) it could not Improved share of exports of 0 >10% be beneficiary firms (15%) measured > 8% during the (10%) project life 6. Increased collaboration n/a n/a Dropped - between Croatian research and 30 See Footnote 4 31 PAD, Annex 3 - Arrangements for results monitoring envisaged monitoring in 2010, beyond the original project closing date. 32 PAD, main text (p. 4) - key indicators section mentions the value of financing, in addition to the number of programs. 20 community and industry with proposed as Croatian Diaspora33 output indicator New 3. SMEs that have new or 7.22% 7% significantly improved products new to the market maintained (as share of total enterprises) source: Community Innovation Survey New 4. Volume of Private R&D €6.0 €9.2 mobilized - through programs supported by STP – in € million (RAZUM, SPREAD and UKF projects with industry) Component A (revised): Encouraging Commercialization of Publicly-Funded Research. Rating: Satisfactory Brodarski Institute: The objective of this component was to strengthen the BIs to promote applied research and commercialize results. BI was supported by the project through assistance in corporate governance, business processes, human resource development, upgrading physical infrastructure and commercialization. The project financed equipment for research projects, consulting services and training, and on a selective basis, incremental operating costs and severance payments for staff downsizing in certain areas. Achievements:  Brodarski Institute was successful in broadening the collaboration with the industry and increasing its share of revenues from private contracts. It concluded 41 research contracts with industry of over EUR 9 million and doubled its share of revenues from private companies from 22% in 2006 to 45 % by end 2010. The share of revenues from new activities increased from 1 percent in 2006 to 48 %. Additional revenues were generated from the participation in four FP7 programs and from international contracts. BI established the Competence Centre Eco Technologies that received support from TEHCRO program.  The initial project objective of restructuring was partially achieved. Although BI reported a self sufficiency ratio of 96% by end project (100% target), the sustainability is still volatile and jeopardized by high costs with the labor force and for maintenance, particularly heating (which in turn is associated to the costs of maintenance of the hydrodynamic laboratory). BI' domestic revenues dropped due to the reduction of contracts with the shipyard industry, itself undergoing major restructurings. Central to restructuring efforts was the governance structure that impeded its transformation into an efficient market-oriented organization that could survive without public funding.  BI‟s infrastructure benefited from investment in new equipments for research, its IT network was established, and new management methods (strategic management, change 33 Annex 3 of PAD does not list how the indicator will be measured, while the main text refers to measured by the number of joint ventures and applied research projects. 21 management, business development, annual business planning and budgetary procedures, and cost accounting) were introduced. BI obtained ISO 14001 certification and it‟s preparing for the implementation of ISO 18001 standards for health and safety management.  BI broadened the cooperation and networking activities on European level. The project contributed to improving project management skills and to changing the behaviors of the researchers towards a more competitive approach to cooperation. As a spillover of BI‟s cooperation agreement with PERA, a UK-based innovation and business support organization, BI participated in four EC‟s FP7 projects with Croatian SME partners. BI became member of European Association of Research Technology Organization (EARTO). In this capacity participated to the FP8 Task force meetings and applied for the EARTO innovation Prize with Fire Rob (autonomous fire fighter robot vehicle).  Lastly, significant cultural and capacity changes occurred in the BI as result of the project. Besides their technical competences, the BI staff acquired project management abilities enabling them to prepare and submit projects for EU funding. More significantly, the need for a deeper restructuring of the institute is now better understood by both management and staff and the boundaries of the needed changes better identified. Rudjer Innovations: The objective of Rudjer Innovations inclusion in the project was to facilitate commercialization of knowledge generated by RBI. RI expended its role and made the necessary outreach strategy, which enabled it also to act as technology transfer intermediary outside the Institute. Box I: Spin-offs funded by Rudjer Innovations: Rudjer Medikol Diagnostics and Rudjer Medicol Cyclotron are two joint ventures between a private health-care provider, Medicol Polyclinic, and the Rudjer Boskovic Institute/Rudjer Innovations. Medicol Cyclotron JP provides highly sophisticated nuclear medicine services for diagnostics of cancer – PET/CT – which has not been available in Croatia. The Croatian Health Insurance Fund committed to co-finance PET tests for the general population, based on international standard criteria. The knowledge of the IRB scientists will be used to operate the cyclotron, which is needed to create radionuclides for the PET. At the same time, researchers will be able to use cyclotron, purchased by Medicol, to do the research work on creating new radionuclides. Rudjer Medicol Diagnostics is a JV with the RBI‟s Department for molecular medicine and is aimed at running genetic testing of hereditary breast cancer. Research studies have proved that certain gene mutations increase the chance of developing cancer, depending on family history. Voluntary screening of persons whose close relatives have developed cancer could establish whether the persons have the gene mutations and possibly increased risk of developing the disease. IRB has already been certified with the European protocol for breast cancer screening. Initium futuri was founded by a group of four undergraduates from the Faculty of Organization and Informatics (University of Zagreb) to provide innovative ICT solutions in telemedicine. Funding provided by RI of EUR 154.000 has enabled establishing the spin off and operational costs of one year. In less than one year the spin off has already signed interesting business contracts and employed an additional staff. The innovations the group generated were successfully presented at European and international trade shows, including at the Microsoft annual Imagine Cup Innovation Program, while in October they were invited to the finals of the 2008 International Federation of Inventors‟ Associations in China. 22 Achievements:  RI was successful in establishing spillover companies and concluding licensing agreements, including with prestigious international institutions. Five spin-off companies were established with project contribution of about EUR 1.5 million: Rudjer Medikol Cyclotron (RMC), Rudjer Medikol Diagnostics (RMD), Intium Futuri, BioZyne and Artes Calculi. Two of these companies are already sustainable, one is in incubation period and the remaining two made initial steps towards sustainability. Four other innovations from RBI where assessed with promising commercial value, however need further development and financing for the pre- commercial phase. 12 licensing agreements, amounting to EUR 750,000 were concluded with clients such as Massachusetts Institute of Technology and Brown University. In addition five patents were granted and 37 patent applications were filed.  The future sustainability of RI is a concern. Such technology transfer institutions need on average 10-15 years to become sustainable, provided that there is solid inflow of innovations coming from the parent research institution. Given its legal status RI is not a state budget user, except through RBI, and could not directly apply for EU IPA grant schemes, only as a partner. University of Zagreb (UoZ) and University of Rijeka (UoR) The objective of UoZ and UoR inclusion in the project in 2008 was to support the commercialization efforts of the two universities, following the establishment of the UoZ Technology Transfer Office (UoZ TTO) and University of Rijeka Technology Office (UoR TTO), respectively.. The TTOs had as objective the development and commercialization of the results generated from the scientific and research activities conducted in the universities. Achievements:  University of Zagreb developed and formally adopted the Intellectual Property Rules TTO book, after broad consultations with component faculties, R&D Committee and various scientific councils. This document defined the procedure for IP protection and commercialization and the revenue-share scheme for licensing. The procedures were tested in about 60 cases that entered into the TTO pipeline. As result, 18 IPs were transferred to the TTO and eight were finally applied for patent protection. In addition, two contracts with the industry were signed.  Four other commercialization policy documents were prepared and are in different stages of consultations before their formal adoption: (i) Guidelines on University Intellectual Property Management; (ii) Guidelines on University spinouts; (iii) Center for Research, Development and Technology Transfer (CRDTT) Rule book; and (iv) Knowledge Exchange Regulations.  UoZ TTO became an important resource for supporting researchers to prepare applications for the Proof of Concept program, managed by BICRO. As result of the services provided to researchers in preparing the project applications, 50 projects were submitted for approval and 20 were approved by BICRO, amounting to about EUR 400,000.  UoZ TTO was successful in promoting commercialization and in raising awareness: it organized more than 60 workshops and other events where about 3,000 researchers participated. The workshops covered topics as IP management, knowledge exchange and entrepreneurship. 23  UoZ TTO launched efforts in establishing the Office as one-stop shop for consultants for various fields and created a database of IPs. It also started a Technology and Expertise mapping exercise, finalized so far for two faculties of the UoZ.  University of Rijeka developed and adopted formally the IPR Rule book for IPs transfer to the TTO UoR. Thus, the mission of the TTO would be to extract the fair market value of the University‟s intellectual property and stimulate the creation of future commercially viable IP using best practices for the benefit of the University and its component faculties. The TTO will closely collaborate with STeP, the University‟s Research Office and Science Managers at key faculties in screening, evaluating and protecting commercially viable innovative projects, technologies, products and services and facilitating the transfer of R&D and new technologies to the business sector. As result, one patent application was filed and two prior art searches were done. In addition, two research contracts were concluded with industry.  UoR TTO was successful in the cooperation with the City Council of Rijeka, which approved the Sustainable Energy Action Plan. This opens further opportunities for collaboration between the university and small companies in the field of renewable energy.  UoR was successful in attracting young researchers in cross faculty knowledge sharing and commercialization of the research results; 97 students submitted 17 projects as result of the competition organized by UoR TTO. Five projects were publicly promoted and presented in media. through the organization of student competition Component B. BICRO: R&D financing program for enterprises (EUR 17.8 mln): Rating: Satisfactory The objective of this component was to increase the abilities of SMEs to invest in R&D, by providing means and appropriate financing mechanisms to support development, use and adaptation of new technologies, promote growth of technology-rich companies, improve quality of the products by industry, which ultimately, will contribute to the improvement of competitiveness of Croatian firms. Financial support was provided through a mix of financial instruments, including matching grants and loans. STP supported the development of one previously-existing BICRO program, RAZUM, and of two additional programs: TEHCRO, and SPREAD. A third new program was initially envisaged in STP, aiming to promote technology commercialization by stimulating the development of venture capital in Croatia (VenCRO). This program was dropped as part of the July 2009 project restructuring. STP also supported BICRO‟s institutional development through direct co-financing of the different industry support instruments and by helping it upgrade its institutional capabilities, including managerial and technical skills, HR system, marketing, business development, MIS, financial management and planning, and monitoring and evaluation functions. RAZUM Program (EUR 8.03 million): The objective of the RAZUM program was to support the development of knowledge-based SMEs with a potential to have a positive impact on economic development. The program provided loans covering up to 70 percent of new product development costs, with maximum support of EUR 1.5 million per project and duration of up to three years. Award decisions were taken by BICRO based on candidates‟ business plans and on the viability of projects. Thus, 24 financing contracts were concluded with 22 firms selected out of 40 firms that submitted full applications (135 outline applications): Box II: Innovative projects funded by RAZUM Service & Resource Management Architecture Software (SRMA) responded to a growing demand for an effective management IT solution for service-oriented businesses operating in highly labour-intensive and knowledge-intensive business environments. With RAZUM support, the Company designed a fully integrated management IT solution which encompasses three platforms: - Knowledge Management System (KMS) - Business Process Management System (BMPS) - Program Management Office (PMO) Not only is the resulting SRMA platform the only product in the market which solves the system integration issue, it is also more cost-effective than combining three separate systems. The importance of such software tools are shown by the fact that Gartner, a leading consulting service, in its regular annual report noted that the development of such tools, as well as their applications in business practices, have recorded permanent growth. The first copy of the solution was sold to NEXE group. SYSTEMCOM (Chip-input Interface for Nanotechnology Biosensors) responded to a growing demand for electronic equipments, particularly for nanotechnology biosensors. The chip developed provides an IP module and/or ASIC solution for the analogue front-end (AFE) function, optimized for current-input, low power & high resolution applications. This chip provides a complete signal path between sensor system and microcontroller, generating a digital output proportional to the input current. It is ideally suited for a broad range of devices with the sensor system - where the generic information about the phenomenon under detection (light or other physical or chemical or electrochemical appearance) - passes the first electrical conversion to the current. This is usually the case in biomedical electronic devices, including battery-powered operation, but also in other “bio� fields as well as in energy & process control / instrumentation / automotive & security / aircraft & military, etc. The IP module implementation enables easy further System on Chip (SoC) integration of follow-up functions. Company Pet minuta Ltd. benefited from RAZUM financing to develop a personal software platform for mobile phones named ShoutEm, which allows social networks to communicate over mobile phones, through high-quality solutions. It requires minimal hardware expense and IT knowledge and is therefore both easy to use and affordable. The product showed such commercial potential after its test phase that Slovenian venture capital fund, RSG Capital, invested $1.2 million to develop ShoutEm into a global product. Five Minutes founders now run ShoutEm in UK too. The company has been operating successfully in Croatia for the past four years and in the past year earned 90 percent of its revenues from exports. Its recent international contacts include the cooperation with major American companies such as Real Network. (www.real.com) Altpro Ltd. is an innovative exporting company whose main business is the development and production of high-technology signaling and safety equipment for railway infrastructure and vehicles. The company used RAZUM financing to develop a product based on so-called axle counters applied in road crossings. The product is based on innovative, security intelligence center-designed modular units. These units satify the so-called SIL4 security level, which is the higest level of security stipulated by the European Committee for Electrotechnical Standardisation (CENELEC). This level of security, combined with the detection system in once single device represents an optimal solution for safe, reliable crossings. Achievements:  135 Outline applications received;  40 Full applications received;  31 projects approved for funding;  22 financing contracts concluded with innovative SMEs;  The total value of the R&D projects amounted to EUR 22 million, of which BICRO‟s contribution was of EUR14.4 million; 25  Total employment in firms was 602, of which 213 were directly involved in the R&D activities;  129 new jobs were created;  Total income realized by end 2010 amounted to EUR 33,8 million;  Net profits generated by end 2010 amounted to EUR 3.42 million.  Increased stock of knowledge capital by the development of 16 new products, 10 functional prototypes of new products; 10 new patents; 6 industrial designs; 5 new processes; 3 publications; 3 improved processes. Additionalities:  Capabilities developed that might influence subsequent R&D productivity: (i) improved firm‟s competitiveness internationally; (ii) increased technical capacity to conduct R&D extended knowledge; and (iii) increased capability of staff.  Expected benefits derived from commercial application of the R&D results: - Increased competitiveness on the international market (60% of firms); - Strengthened presence on the targeted markets (65% of firms); - Improved knowledge of the markets (75% of firms have good or excellent knowledge); - Improved skills for commercialization (15% of firms with excellent skills and 75% of firms with good or medium skills)  Changes in firm behavior that resulted from having received support from RAZUM:  New innovation ideas were generated: 55 percent of the firms generated new ideas while working on the project, but did not have enough capacity to work on other innovations and to engage in additional R&D activities and 35 percent managed to work on additional R&D activities in parallel with the project itself.  R&D human capacity increase appeared to be sustainable: 80 percent of the firms intend to keep the newly hired staff after the project closure. SPREAD (IrCRO) - Sponsored Research and Development) Program (EUR 2 million) The objective of the SPREAD program was to encourage SMEs to invest in R&D, facilitate ties with R&D institutes (RDIs) and universities and help the latter expand their industry network and outreach. The SPREAD program was in essence a risk sharing scheme designed to address the market failure of underinvestment in R&D and the lack of trust and collaboration between industry and the knowledge community. The program funded up to 50 percent of the cost of joint projects through non-reimbursable matching grants to develop or adapt new products or processes, with a maximum contribution of EUR 80,000. Achievements:  22 firms were selected to implement R&D projects;  The total value of the projects amounted to EUR4.72 million, of which the project financed EUR 2 million.  Increased stock of knowledge capital by the development new products or new services ( 88% of firms); improvement of products (33% of firms), new processes (22% of firms) 26 Additionalities:  Expected benefits and behavioral changes derived from the project: - Affordable financing of projects (72 % of firms); - Improved competitive position on the domestic market ( 61% of firms); - Improved competitive position on the international market (56% of firms); - Increased internal capacity for R&D activities and enhanced reputation and image (39%); - Strengthened collaboration and networking with new partners (28% of firms). - Commercialization of products started for more than 60% of firms. Box III: Innovative projects funded by SPREAD (IrCRO) INTECO - Robot for hydrodynamic treatment of cement and metal surfaces. The project financed the development of an advanced prototype of a robot for hydrodynamic processing of concrete and metal surfaces and of additional, attachable tools. While the basic prototype of the robot was developed prior to the project, the development of the advance prototype required a close cooperation with the scientists from the Laboratory for Robotics and Intelligent Control Systems of the Faculty of Electrical Engineering and Computing, University of Zagreb. The utilization of the robot is mainly in construction (the restoration and preparation of cement surfaces) and shipbuilding (preparation of metal surfaces) activities. The advantages of the solution lie in its electrical propulsion and its small and dismountable, lightweight and user-friendly design. It can function autonomously once placed on site with its built-in system for self-guidance. Racuni.hr Ltd is specialized in the field of the development and implementation of e-bills and e-file systems. Through their system, users can easily manage creating and sending digital bills for efficient payments. More than 10 million bills are delivered in Croatia alone, and most of these are delivered physically, on paper. SPREAD financing allowed the company to develop data processing platform for e- bills with ability of hosting services and frames. The project end-result would allow facilitating the invoicing procedure, minimize mistakes, provide better overview of cash flows, while being environmentally fiendly for citizens and public institutions. TEHCRO Program: Technology Centers, Science, Incubators and R&D Centers (EUR 7 m) The program aimed to catalyze commercialization of R&D and growth of technology-based local businesses through support for projects aimed at the establishment, upgrading or development of: (i) Technology Business Centers, supporting establishment and growth of technology-oriented, innovative and knowledge-based companies at the regional level; (ii) Science & Incubation Centers, supporting the incubation of innovative academic spin-offs and start-ups; and (iii) Research & Development Centers: supporting R&D within specific thematic topics. TEHCRO provided grant funding for a maximum of 65 percent of the project's operational costs (or 50% of total costs) selected by public tenders or on BICRO‟s initiative. Aplicants were invited to submit project proposals in the form of business plans. In order to be eligible for funds, they should have been (not-for-profit) legal persons or limited liability companies established in Croatia. Only projects targeting financial sustainability within a reasonable timeframe and managed by private sector entities (NGOs or private entrepreneurs) were eligible for support. Achievements:  13 TEHCRO Centers signed contracts to participating in the program: Varaždin Technology Park, Science & Technology Park-University of Rijeka, Mariculture Business Innovation 27 Center of the University of Dubrovnik (MARIBIC), Business Innovation Support Center of the Industrial Park Nova Gradiška, Technology Innovation Center Međimurje of the Regional Development Agency Medimurje (TICM), TERA Tehnopolis of the University of Osijek (TERA), the Center of Competence for Eco - Technology of the Brodarski (CET), Competence Center for Renewable Energy Sources (CC OIE),etc.  Total BICRO‟s investment amounted to EUR 10 million; each Euro invested through the TEHCRO program attracted in average EUR 1.34 from other funding sources.  Funds were used as follows: 24% in infrastructure, 39% in specialized equipment, 37% in services for business development and technology transfer;  146 training courses were delivered;  115 new firms were incubated;  286 new jobs were created in tenant companies between 2008-2011 Additionalities:  85 products and services were developed by the firms;  33 prototypes were developed;  7 patents were applied. Box IV: Innovative projects funded by TEHCRO Mariculture Business Inovation Center of the University of Dubrovnik – MARIBIC Established in June, 2008 as part of the TEHCR MARIBIC‟s main purpose is to become a missing link between scientific/educational institutions and active and prospective aquaculture producers, taking a vital role in the advancement of practical education and student research, and in the development of the aquaculture industry. MARIBIC integrated fish production system comprises: broodstock conditioning facility, finfish hatchery, recirculating aquaculture production systems and floating cages. Shellfish hatchery consists of brood stock conditioning systems, facilities for larval and spat production and laboratory for phytoplankton production. Within the scope of its technology-innovation activities, MARIBIC works towards the development of new technologies, the evaluation and transfer of existing technologies, development and evaluation of new equipment, and on the introduction of new species and new products in aquaculture. Business-incubation activities included support to new companies in all initial phases of the organization: development of a business plan, contacts with financial institutions, production planning, technology transfer, and acquisition of an initial work environment. By organizing seminars and presentations by experts in the various fields of aquaculture, MARIBIC provided education and information about new technologies, trends, equipment, marketing, and about the problems shared by the local aquaculture industry and aquaculture producers, particularly in view with harmonizing to EU market requirements. The center was also active in the scientific field, working with University of Dubrovnik and other scientific institutions, local governmental institutions and producer‟s organizations on scientific and development programs. Proof of Concept Grants Fund (PoC) (EUR 1.3 million): The key objectives of the PoC were to provide innovative companies and researchers an opportunity to verify and validate commercial viability of research results and establish an appropriate strategy for continued commercialization. The program was administrated by TEHCROs centers and provided grants on a competitive basis of up to 75% of total project costs, supporting external expenditures on pre-commercialization proof of concepts activities. 28 Achievements:  73 contracts were signed out of the 238 full applications received;  The total financing from BICRO amounted to EUR 1.3 million; an additional EUR 0.5 million were the contribution of the private sector and in king contributions from the research organizations;  29 patent were applied;  19 technical feasibility demonstrated;  40 prototypes realized. Box V: Innovative projects funded by Proof of Concept Azonprinter Ltd received financing for its project „ProChecking the innovative concept of the machine for digital printing on all materials for industry“. It is about a new technology that enables printing with different types of inks on objects of various sizes and functionalities with different kinds of surfaces, such as heating cells or photocells for solar power generation. The project helped developed a prototype machine for digital printing that by using environmentally friendly inks enables print on all materials, without any preparatory and coatings. This is an improved technological solution because it allows the printing of very high quality, small series, a large number of mutations, environmentally friendly inks with no preparations object to which the press. After this phase, Azonprinter will go into commercial production and sales of technical solutions through a network of existing international dealers. The above technology has a great application in the glass industry, aluminum, and other designs. Veski Ltd benefited from PoC financing for its project „Device for continuous monitoring of wind turbines with integrated control functions of mechanical and electrical parameters“. The concept of this product is based on the implementation of an algorithm for continuous monitoring of the drive generator status (measurements of electrical parameters), in combination with conventional measurements of mechanical parameters. The innovation consists of an application and linking of information on one single device (as compared to a more expensive and complex solution of two or more devices). This represents an improved technology of monitoring the wind gernerator and incorporating the wind energy with different measurements. The project developed the algorithm that can provide concise information management systems and facilitate the management of aggregate, increased security and efficiency. After successful testing of the model, the company completed the prototype device, filed a patent, and is developing its commercialisation plan. Component C. Unity through Knowledge Fund (EUR 5.55 million): Rating: Highly Satisfactory The objective of this component was to strengthen research collaboration between Croatian scientists in the country and international community, in particular Diaspora, through provision of grants for joint scientific projects, as well as targeting research activities among young scientists. Specific programs were developed and implemented to better target the beneficiaries: (i) Cooperability Program: supported joint research projects of Croatian scientists in and outside of Croatia with the goal of attracting prospective scientific research in Croatian companies and institutions. The projects financed within this program are large collaborative research projects of up to EUR 200,000 and implementation duration of 2- 3 years. Two subcategories of projects were developed: “Homeward� and “Crossing Borders; (ii) Connectivity Program: encouraged international mobility of professionals and scientists in order to enhance the flow of knowledge and skills in Croatia. The program could finance projects up to 10,000 EUR and with implementation duration of 6 months. Two subcategories of projects were developed: “Gaining experience� and “Homeland visit�: (iii) Young Researchers and Professionals Program: 29 encouraged professional advancement of Young Researchers, with an emphasis on establishing cooperation with international institutions and the economic sector, in order to enable them to lead scientific research autonomously. The Fund could finance projects up to 100,000 EUR and with implementation duration of maximum two years. Three subcategories of projects were developed: “My first research,� “Reintegration,� and “Research Industry.� Box II: Innovative projects funded by UKF Applied Neuro-Fuzzy Control of Turbine Governors and Ship Rudder Systems. Goals of the project were to establish laboratories for advanced industrial regulation systems in Brodarski Institute, transfer of knowledge from the field of advanced regulation systems and development of regulation algorithms with neuron networks and fuzzy control algorithms for turbine governor and ship rudder systems. The laboratory has been established and is being used regularly, for scientific research and commercial projects. It has been used in projects in HE Lešće and HE Čakovec, which was created by the Brodarski Institute. The knowledge gained with the use of the laboratory equipment has been used in the development and activation of the wave generator regulator, a device that creates waves in test pools for testing ship models for sailing conditions in wavy waters or for testing port profiles in wavy waters. Physiology of SCUBA diving (Young Researchers and Professionals Program). The project collaborated with Diving Alert Network, Norwegian University of Science and Technology and other partners. The results of this project were partially used for the application to the EU Framework 7 programme in the Marie Curie Initial training network category with Split group as partners. Other participants include several partners from industry and SME, including one large international company, MARES. Based on the results collected as part of the UKF project, a grant application was submitted to the US Navy (Office for Naval Research) and based on the first evaluation, a one year funding is to be awarded. This way, the group from the Department of Physiology at Split School of Medicine will become the second research team (besides investigators from Israel) outside of the United States that is funded by the US Navy. Study and development of ultrasonic non-destructive examination systems (SONDE). The aim of the project is to set up an ultrasonic applications research group focused on design, development, production and testing of "made in Croatia" customized ultrasound probes for non-destructive evaluation of structural health in nuclear and fossil fuel power plants. At present, several prototypes have been finalized and tested. One set of the final products is planned to be sent to Slovenia for performance demonstration purposes, and another to be used in an inspection in Hungary. The project leader returned to Croatia upon receiving the UKF project. The project collaborates with several local and international institutions including the Swiss Federal Institute of Technology, Georgia Institute of Technology, and University of Zagreb. Other results include publications, training of PhD students, and several patent applications. Research, Development and Construction of portable micro X-Ray Fluorescence Device The goal of the project was development of a prototype of a portable instrument for the microanalysis of x- ray fluorescence (micro-XRF), which in many aspects would be superior to similar high-quality devices currently available on the market. The potential use of such a versatile instrument covers a wide field of use in archaeometry, museology, the chemical industry, environmental research, forensics, metallurgy, etc. The original design developed within the project was a transfer of a combination of scientific and technological knowledge, which was acquired by the team leader abroad, and the mutual efforts of the team of Croatian professionals, who are prepared to transform their theoretical and practical knowledge into a unique, high- technology product. Achievements:  91 projects were implemented: Cooperatibility: 26 projects, Connectivity: 42 projects; Young Researchers: 26 projects  UKF funding totaled EUR 5.1 million (Cooperatibility: EUR 3.7 million; Connectivity: EUR 287,000; Young Researchers: EUR 1.07 million) 30  Additional EUR 1 million and EUR 784,000 were mobilized by the foreign research institutions and the Croat institutions participating in the projects, matching the UKF funding of EUR 5.1 million  544 researchers and 260 institutions collaborated in the projects  166 foreign researchers, of which 41 scientists from Diasporra and 133 reputable foreign basic scientific research institutions, as Stanford University; Mack-Planck-Institute for Molecular Cell Biology and Genetics, Institute of Biochemistry II Goethe University Medical School; Swiss Federal Institute of Technology Lausanne (EPFL), Swiss Federal Institute of Technology Zurich (ETH); Fred Hutchinson Cancer Research Center, KTH Royal Institute of Technology and Johns Hopkins University, USA, Laval University, Canada, Academy of Fine Art, Austria, Norwegian University of Science and Technology, etc.  53 projects, amounting to EUR 2.5 million, involved cooperation with industry and mobilized an additional EUR 655,000 from private funds Component D. Project Implementation and National Innovation Support Group (EUR 2.1 million) Rating: Satisfactory Implementation Unit Support (EUR 1.2 million). The subcomponent was to finance the related costs of operating the Project Management Unit established in MSES, including staff, financial management system, audits, equipment, training, technical assistance and studies. Croatian Innovation System Group and Croatian Accreditation Council (EUR 0.6 million, of which EUR 0.44 million from the loan) The subcomponent was to finance costs associated with the operations of the Croatian Innovation System Group organized by MSES and the Croatian Accreditation Agency. Eligible expenditures for this component included consultants‟ services and training. Original results for components and indicators Revised results for components and indicators Component A: Restructuring RDIs Component A: Encouraging commercialization of publicly-funded R&D Original Result: Restructuring of participating Revised Result: RDIs and TTOs commercializing publicly-funded R&D R&D institutions (at least 2: BI and 1 more RDI) Baseline Target 2009 Baseline Target Original indicators Change Revised or new indicators 2006 (2010) 2006 2011 % of planned 0 75% (100%) Revised during BI‟s share of revenues from 22% 35% investments implement. to private companies in % completed better measure (disbursements) project activities and objectives Number of applied n/a n/a Revised due to - Number of IP taken to protection 0 60 research projects changed from RBI and third parties by RDIs objective - Number of Patents Granted to RI 0 6 - Number of IP transferred to 0 20 Zagreb and Rijeka‟s TTOs (cumulative) 0 8 - Number of patents filed to the TTOs (cumulative) 31 Component B: Upgrading technology financing Component B: R&D financing program for enterprises programs Revised Result: Increased ability of SMEs to invest in R&D activities Original Result: Technology financing programs (at least 3 - RAZUM, Venture Capital and SPREAD) started and operational Baseline Target 2009 Baseline Target Original indicators Change Revised or new indicators 2006 (2010) 2006 2011 Amount of $10 mln. 43.8 (58.3)34 Revised due to Amount of financing provided (€ financing provided removal of VC mln) €18.03 €33.68 (€ mln) - RAZUM program (€ mln) 0 €1.5 - SPREAD Program (€ mln) 0 12 - TECHCRO program as number of projects Number of 25; 0; 10; 60 (70); 3 (6); Revised due to Number of SMEs directly 37 78 enterprises 0 50 (60); 30 removal of VC35 supported by programs supported (35) (cumulative) by RAZUM, VC, TCs & SPREAD Component C: the Unity through Knowledge Component C: the Unity through Knowledge Fund (UKF) Fund Revised Result: The UKF operational on a sustainable basis Original Result: The Fund becoming operational and financing eligible projects Baseline Target 2009 Baseline Target Original indicators Change Revised or new indicators 2006 (2010) 2006 2011 Number of projects 0 50 (60) Kept as original, Number of projects (total) 0 5037 supported through with new - Number of projects involving 0 15 the Fund indicators added cooperation with the industry 36 (cumulative) - Number of projects with young 0 15 scientists 0 30 - Number of projects successfully completed (as evaluated by UKF evaluation committee) Number of 0 30 (30) Kept as original Number of projects with Croatians 0 30 Diaspora scientists living abroad receiving funding Component D: Project implementation Support Component D: Project implementation support and National Innovation System and Croatian Innovation System Revised Result: Consolidation of the institutions and policies of Croatia‟s Original Result: n/a National Innovation System 34 Target was defined during project implementation. 35 SPREAD financing cap has been increased to adapt to beneficiary needs, resulting in fewer projects, and a decreased target. 36 This indicator includes projects primarily oriented to industry and other UKF projects where industry nvested matching funds. 37 UKF target has been lowered due to increased share of projects of higher value as compared to medium-value projects. 32 Box VI. Amendments to project components Reallocation Letter; Level 1 Restructuring: July 9, 2009 Restructuring Paper: Report no. 49457; Amendment Letter July 31, 2009; Level 2 Restructuring: Dec 7, 2010 Restructuring Paper: Report no.58376; Reallocation of Loan proceeds Letter Dec 15, 2010; Level 2 Restructuring: Mar 31, 2011Restructuring Paper: Report no.61305v1; Reallocation of Loan proceeds Letter, May 19, 2011; Level II Restructuring: May 15,2011Restructuring Paper Report no: 61971v1. November 2007 Amendment Letter introduced two changes: (i) the ventures supported by RI for the purposes of commercializing (“Beneficiary Ventures�) became eligible for financing; and (ii) financing of the specific projects developed by the “beneficiary ventures� for the commercialization of intellectual property innovation (licensing of intellectual property and the establishment of spin-offs) was allowed. EUR 2.6 million were reallocated to a new expenditure category (“Innovation financing�) and earmarked to RI. These changes were not considered as requiring project restructuring and were approved as reallocation of loan proceeds. Box VII. Benchmarking of BICRO and RBI components Project instruments were tested by benchmarking eleven BICRO financed companies and RBI start-ups with their U.S. counterparts. A set of U.S. companies that are recipients of public funds through Small Business Innovation Research Program and Small Business Technology Transfer were compared to recipients of BICRO funds and RBI start-ups. Comparison showed that all entities demonstrated a market and business vision, and product/service status comparable to their U.S. counterparts, given their stage of development. Both institutes were employing a good quality screening mechanism to solicit, assess, and deploy funding support to high technology, innovative companies of merit that have the potential to grow. The funding provided by these programs was instrumental in advancing the development of these enterprises and sustaining their businesses during the critical periods inherent to early stage enterprises. Source: STP Supervision Mission Aide Memoire, February 2011 33 Annex 3. Economic and Financial Analysis The project is assessed to have achieved a high level of efficiency. The PAD justified the efficiency of the project based on (i) results from developed countries that showed that the rate of return on R&D investments is around 20% and on (ii) the public good nature of the intervention. Cost-benefit analyses are however not easily applicable to evaluating efficiency of the project for a number of reasons: i. The gap between social (economic) and private rates of return – the latter measured by internal rates of return (IRR). ii. IRR in the present case would also be highly dependent on the values assigned to the results of commercialization activities (especially the value of patents and of spinoff companies) iii. Intangible benefits related to capacity building (corresponding to a non-negligible share of the project), as for instance in the case of UKF‟s invitation to start negotiations for the FP7 COFUND project (EUR 7 million would be available in next five years). In the absence of standard cost-benefit analyses, we follow ICR Guidelines 38 and use two different methods to evaluate project efficiency, through estimates of (a) social opportunity costs, (b) cost effectiveness. A third method assessing the outcomes of the RAZUM program (in terms of research capability and job creation) is presented in Box A.3.3. Estimating social opportunity costs To estimate the social opportunity cost of investments in R&D, we compare the rate of return on R&D with other possible public investments in Croatia. For practical reasons, we chose to limit the comparison to investments in education and infrastructure – the bulk of the projects of the country‟s 2010 Economic Recovery Program. Using standard techniques, we computed the rate of return on R&D and infrastructure investments in the country and used results of the literature for the returns on education in middle income countries (Box A.3.1). Our findings show that estimated rates of returns on R&D in Croatia (73 percent) are at least double the value of returns of infrastructure (around 24-34 percent) and seven times higher than in education (around 10 percent – or ten times higher if the rate of return for schooling for high income countries (7.4 percent) is used). Results for R&D and infrastructure are consistent with the findings of the economic research. Higher returns for R&D in Croatia are consistent with the relatively lower stock of R&D capital vis-à-vis the stock of infrastructure and human capital in the country. 39 Box A.3.1. Macro Level Analysis on Social Returns to R&D, Infrastructure, and Human Capital Using World Development Indicators macro level data, we compute the rate of return on gross R&D investment. Following the methodologies used in Coe and Helpman (1995), Jones and Williams (1998), and Lederman and Maloney (2003), we first construct R&D capital stock and then find the elasticity of various output measures with respect to R&D capital and computed gross rate of return (see table below). Rate of return on R&D investment for GDP measured in constant LCU is 73%. Based on these findings, optimal amount of R&D investment can be estimated. In Croatia this amounts to 9.23% of GDP. Real interest rate in Croatia between 1997 and 2010 is around 38 ICR Guidelines, Appendix G 39 Seker (2011) 34 7.3%. Current gross rate of return is 0.73% which is close to 10 times less than real interest rate. As a result, optimal amount of R&D expenditure would be 0.92*10=9.2% where 0.92 is the average R&D expenditure (as a % of GDP). Using macro level data, there are several studies that estimate the rates of return to R&D. Among these studies, Lederman and Maloney (2003) use cross-country data for 1975-2000 period and find that returns to R&D investment is around 78%. They group countries according to their income levels and find: 20-40% as OECD average, 60% for medium-income countries, and around 100% for poor countries. Returns to R&D in developing countries are higher than the values for industrialized countries. Coe and Helpman (1995) estimate rates of return to R&D for the period 1991-1990 and find 123% for the G7 and 85% for the remaining 15 OECD countries. Lichtenberg (1994) works with a cross section of 53 countries and argues that the return to private R&D investment is seven times larger than to fixed investments. The result for rate of return obtained for Croatia is in accordance with the studies in the literature. Elasticity and Rate of Return of R&D Investment Optimal Amount of R&D Investment Gross Rate of Real interest rate Lending interest Elasticity Return (in %) (%) rate (%) GDP (current LCU) 1.26 139 9.23 5.64 GDP (constant LCU) 0.66 73 GDP (constant 2000 US$) 0.66 73 GDP per capita (constant LCU) 0.66 73 GDP current LCU/Labor force 1.34 147 GDP constant LCU/Labor force 0.75 83 Canning and Bennathan (2003) compute the rate of return to infrastructure (electricity generation and paved roads). They find that return to electricity generation capacity is 50% where the values vary significantly across countries. To give some examples, in Turkey, rate of return is 32%, in Yugoslavia 24%, Portugal 7%, Mexico 51%, and Brazil 10%. They find the rate of return to paved roads as 30%. In their study, elasticities of GDP per worker with respect to electricity services is 9% for a country at median income level and 6% for a country at the lower quartile. Elasticity of GDP per worker with respect to paved roads is 9% for a median income level country and 5% for a lower income level country. In a recent study Drezgic (2008) estimates the elasticity of GDP across Croatian counties with respect to transport and electricity sectors. The elasticity varies around 4.7% and 4.6% for transport and electricity sectors, respectively. In another specification where he combines transport, electricity, and construction sectors, elasticity of GDP with respect to this combined sector is 6%. Based on these estimates, rate of return on infrastructure in Croatia can be estimated using the methodology used in Canning and Bennathan (2000). � � � Consider Y  AK L X as the Cobb-Douglas production function where K is physical capital, L is labor and X is infrastructure capital. Solving the aggregate production function, we get the following first order conditions rK p X Where px is rate of return to infrastructure capital and r is the rate of return to physical capital. � ,�  x Y Y Drezgic (2008) compute net capital stocks for each industry in Croatia from 1996 to 2006. He shows that electricity sector makes around 19% of total private capital stock (X=zK where z is a multiplier). Transportation sector makes 21% of total private capital stock. Return to private capital stock which is the real interest rate is around 7.3%. Using this information average rate of return on electricity and transportation sectors can be calibrated as follows. �Y �Y �Y �r px     X zK �Y z� z r According to this formula and using the estimates obtained from Drezgic (2008) rate of return to infrastructure is 34% for electricity and 32% for transportation, and 24% when electricity, transport, and construction sectors are combined. The literature on rates of return to human capital is much older. Psacharopoulos (1994) estimate the social returns to investment in education level for a large number of countries. An updated study which presents results for the same countries in Psacharopoulos and Patrinos (2004) find that average rate of return to an additional year of schooling is 9.7%. Returns for low, middle, and high income countries are 10.9%, 10.7%, and 7.4% in respective order. The value for Croatia is likely to be around 10.7%. These findings on rates of returns on R&D, infrastructure, and human capital show that the returns to R&D (73%) are quite significant and higher than the returns of infrastructure (around 24-34%) or human capital (around 10%). Source: Seker (2011) 35 Assessing cost effectiveness We also analyze project efficiency using the notion of cost effectiveness to determine whether the project represented the expected least-cost solution to attain identified and measurable benefits. In order to assess the efficiency of the projects, one must ask a hypothetical counter- factual question, namely: Would the two development objectives be achieved without the project support? The idea is that a negative response to this question would imply a high level of efficiency of the project for it generates an outcome otherwise inexistent with a finite budget (total cost of the project). By providing technical assistance to TTOs fostering collaboration with the business sector, the project enabled research institutions to commercialize research outputs that would otherwise be idle. Research commercialization is one of the central challenges for increasing the impact of R&D investments on economic development in Croatia and other transition economies and was essentially non-existing prior to implementation of STP, according to the opinion of local experts and scientists. By subsidizing the collaboration between enterprises and public research organization, the SPREAD program (a classical matching grant scheme) contributes to reduce the risk of the partnership, perceived by both parties as high due to reputational factors – a supposed poor quality of research in public research organizations and supposed irrelevance of research proposed by the private sector. The results of the program may be preliminarily inferred from a survey implemented by the Zagreb Institute of Economics (ZIE) (Box A.3.2): enterprises beneficiary of SPREAD programs seem to, on average, agree with the statement that the collaboration would not exist otherwise (average score: 3.5).40 Firms also believe that without such collaboration, the same results would be achieved within a longer period and the scope of the research would be significantly reduced (Table A.3.1). While other factors may affect the likelihood of research commercialization, no other relevant events strong enough to alter the behavior of scientists and businessmen are known and the overall incentive regime remained the same. Similarly, by providing financing for business related R&D through Proof of Concept (small grants to finance early stage financing) and RAZUM (a conditional loan for later stage innovation) the project seems to have generated positive additional effects on SMEs‟ ability to invest in R&D – rather than substituting for private investments that would have occurred in the absence of the public intervention. BICRO‟s programs target the so-called “Valley of Death� – an early stage development phase of startup companies for which financing is known to be less available.41 In addition to overcoming this structural factor, BICRO‟s program helped to mitigate 40 On a scale ranging from 1-strongly disagree to 5-strongly agree. Source: Zagreb Institute of Economics. 2011. Evaluation of the Innovation Programs Financed by the World Bank in Croatia. 41 The market failure for funding early stage ventures is related to information asymmetry and moral hazard problems that are classical in financial markets but particularly relevant in the case of early stage financing (see Lerner, J. and Hall, B. (2009) “The Financing of R&D and Innovation,� NBER Working Paper No. 15325). Since the innovator has the information on the success of its project which the investor lacks, a financial gap is formed between the two parties. The funds provided by BICRO and research institutes alleviated these market failures and led to successful spin-offs and commercialization of innovative ideas. 36 the liquidity constraints for business investments in R&D generated by deleveraging of financial markets and increased risk aversion of investors in the wake of the global crisis.42 The results of RAZUM may also be preliminarily inferred from the ZIE survey: In the absence of program funding, a third of companies would not have been able to start activities, while others would have tried to plough on mostly relying on their own resources. Consequently, product development would be slower and most likely scaled down. Potential market advantage would be diminished or lost. Smaller budget would have affected the duration of the project (would have been longer), scope of the project (smaller), R&D capacity through additional employment of R&D staff (lower), and innovativeness level of the project (also lower). A similar evaluation was made by OECD for EU and Austria programs, reaching comparable results (Table A.3.2). A similar result was achieved for the UKF Cooperability Program. The absence of funding would have lowered the level and quality of scientific and/or technological output, would have resulted in fewer papers published in top ranked international scientific journals, and would have precluded access to necessary research equipment. Furthermore, it would have been harder for researchers to develop cooperation with researchers of the Croatian Diaspora and with foreign scientists, as well as to educate young researchers abroad (Table A.3.1). Another important result that helps to understand the impact of UKF in terms of research excellence: projects supported by UKF were two times more likely to obtain additional funding from FP7 than the average Croatian project application. Table A.3.1: Additionality by SPREAD and UKF programs Type of Behavior Additionality Anticipated effect to failure to receive government funding Accelerated Expanded scale and Improved More challenging Project Launch Schedule scope research base research SPREAD Do not Longer duration of Smaller scope of R&D Do not Program implement R&D (4.4) (4.5) collaborate with Beneficiaries project (3.4) RI/Univ (3.5) Do not invest in R&D (3.6) UKF Program Do not complete Difficulties to Lower scientific, No cooperation Fewer publications in Beneficiaries the project (4.5) establish cooperation technological output with expatriate top ranking with foreign (4.4) researchers international scientific scientists (3.8) (3.2) journals (4.0) Lower level of No access to No opportunity to scientific, technological needed research educate young quality (4.0) equipment (4.0) researchers abroad (3.9) * Items were measured on the scale ranging from 1-strongly disagree to 5-strongly agree. Values in parentheses show the average scores based on beneficiaries responses. Source: ZIE (2011) 42 One important effect of the crisis is that cash deprived firms had to use more internal funds to finance working capital thereby reducing the availability of internal funds for new investments, including for R&D (see Correa and Iootty (2010)). 37 Table A.3.2: Additionality by the RAZUM program Type of Behavior Additionality Anticipated effect to failure to receive government funding Accelerated Expanded scale Improved research More challenging Project launch schedule and scope base research RAZUM 25% would have 65% would have 60% would have 20% would have 40 % would have Project gone with another taken longer reduced the scope not had appropriate been less innovative beneficiaries research; equipment; 55% would have 10% would have not hired new cancelled people EU (5th 10% would have 33 % would have 75 % would have 43 % would have Framework cancelled taken longer reduced the scope been less challenging program) Source: OECD Austria (FFF) 28% would have 32% would have 74% would have 49 % would have Source: OECD cancelled taken longer reduced the scope been less challenging Source: ZIE (2011) Box A.3.2. STP Assessment by the Zagreb Institute of Economics – Key Methodological Aspects The assessment of STP is based on the surveys conducted to beneficiaries of three programs, RAZUM and SPREAD (from BICRO) and UKF‟s Cooperability Grants. Based on preliminary interviews with some of the beneficiaries of these programs a survey was designed and implemented on a large sample of firms/scientists. For RAZUM program the survey-participants included both recipients of funds and firms that are either still in the evaluation process or are approved and waiting for financing. For SPREAD program firms that are recipients of SPREAD and have begun to work on the project were included (there were no pending projects for SPREAD). For UKF program both successful and unsuccessful applicants were surveyed. The goal of these questionnaires was to find out to what extent the recipients benefited from program funding and in case of UKF it also intended to show to what extent did the non-recipients “suffered� because they were turned down for the funding. They were sent to the entire populations of recipients of RAZUM, SPREAD and UKF Across the Borders Grant. Population of RAZUM recipients consisted of 22 firms of which 20 responded to the questionnaire. Out of 24 firms waiting for RAZUM funding, 14 responded to the survey. 18 out of 20 recipients of a SPREAD grant participated in the study. UKF Cooperability Grant financed 26 projects and 20 successful applicants responded to the survey. Among unsuccessful applicants to UKF, 15 out of 79 responded to the questionnaire. Questionnaires were sent to managers/scientists that participated directly in the projects. The study measured input, output, and behavioral additionality created by the programs. Output additionality deals with the extent to which the firm‟s output has changed as a result of having received a subsidy. It also captures the increments to firm‟s stock of knowledge capital resulting from the R&D project and the benefits derived from commercial application of R&D results. On the other hand, behavioural additionality deals with changes in firm behaviour that resulted from having received public support. It includes the changes in the breadth of innovation activities and changes in both technological and business strategies of the firms. To get the complete picture of subsidy effectiveness, three components of additionality were used together to determine whether the R&D would have been carried out without public support, whether the public funding changed the scale and scope of the R&D, or whether the R&D would have been done differently. To explore each of the three components of the concept, responses were collected on five- point Likert scale and multiple choices questions. To assess the additionality, questions were asked involving a “counterfactual scenario�, i.e. a hypothetical situation where recipients had to imagine they had been denied the grant. Source: Zagreb Institute of Economics. 2011. Evaluation of the Innovation Programs Financed by the World Bank in Croatia 38 Box A.3.3. Assessing outcomes - research capability job creation: The case of RAZUM Methodology: In a 2011 study, Radas and Anic apply qualitative research as well as quantitative analysis based on a survey of selected RAZUM beneficiaries, following the methodology of the Small Business Innovation Research (SBIR) program in the U.S, and assessing input, output and behavioral additionality. The additionality concept in this study was adopted from Georghiou (1994), while treatment of particular aspects of additionality was adopted from Clarysse et al. (2009) and Hsu et al. (2009). Following the long history of subjective measures of the effects of R&D and innovation grants, the data on additionality was collected through self-assessment: such measures are widely used and accepted in the field of evaluation research (Clarysse et al., 2009). Following extensive literature survey in Hsu et al. (2009), and taking into account specificities of the RAZUM program Radas and Anic (2011) defined how to measure the three dimensions of additionality (Table A.3.1). As in Hsu et al (2009), assessment of additionality was made through questions involving a “counterfactual scenario�, i.e. a hypothetical situation where recipients had to imagine being denied the grant. Items used to assess input, output and behavioral additionality Input - Scope and scale of the project additionality - R&D intensity before and after the grant Output - Number of new products/services introduced to the market additionality - Number of new products/services under development Behavioral - Innovation development process additionality - New employment - Collaboration Findings: In terms of input additionality, data showed that in absence of RAZUM funds, companies would have received a lower budget, whereas activities would have been conducted on a smaller scale as well as in a slower pace. Meanwhile, R&D intensity increased after start of the RAZUM program. By comparing firm‟s R&D intensity before and after the start of the RAZUM grant, in 9 out of 15 companies, an increase in own R&D intensity was observed after start of the grant (on average from 0.41 in the year before RAZUM grant to 0.44 in the year after the beginning of the RAZUM grant). The effect of RAZUM on output additionality can be seen in firms‟ innovation stock. The observed increase indicates positive effect of the grant on firms‟ innovation pipeline. An interesting question is whether this increase results from spillover of ideas from RAZUM to other projects. Majority of firms (17 out of 20 firms) reported that they indeed generated ideas for new product/services during the RAZUM grant, although mostly they did not have capacity to work on them. This indicates that the increase in the innovation pipeline reported in table 5 was not caused by spillover ideas from RAZUM. Considering the magnitude of the increase, it is most likely caused by the RAZUM product itself. RAZUM also generated new employment (54 new jobs), most of which has been in R&D (48 jobs). The firms expressed their intention to keep the new staff after closing of the grant, which suggests that firms anticipate permanent increase in cash flow as a result of RAZUM (Wallstein, 2000). Moreover, without RAZUM funds, firms would have been unable to hire new staff for the project and no new collaboration would have been established. Since collaboration and employment of skilled staff raise R&D capacity of the firm, without RAZUM the companies would have been unable to increase their R&D capabilities in the same way. The influx of knowledge coming through collaborative networks would have also decreased. As expected, collaboration with firms would have been affected less than collaboration with research institutions. Source: Radas, Sonja and Anic, D. (2011) “Additionality of an innovation subsidy targeted at SMEs� Zagreb Institute of Economics Working Paper 39 Table A.3.3. Average Innovation Performance Sub-indicators CROATIA Project Effects ENABLERS Human resources 1.1.1 New doctorate graduates ↑ Direct 1.1.2 Population completed tertiary education 1.1.3 Youth with upper secondary level education Open, excellent and attractive research systems 1.2.1 International scientific co-publications ↑ Direct 1.2.2 Scientific publications among top 10% most cited ↑ Direct 1.2.3 Non-EU doctorate students ↑ Direct Finance and support 1.3.1 Public R&D expenditure ↑ Direct 1.3.2 Venture capital FIRM ACTIVITIES Firm investments 2.1.1 Business R&D expenditure ↑ Direct 2.1.2 Non-R&D innovation expenditure ↑ Direct Linkages & entrepreneurship 2.2.1 SMEs innovating in-house ↑ Direct 2.2.2 Innovative SMEs collaborating with others ↑ Direct 2.2.3 Public-private co-publications ↑ Direct Intellectual Assets 2.3.1 PCT patent applications ↑ Direct 2.3.2 PCT patent applications in societal challenges ↑ Direct 2.3.3 Community trademarks ↑ Direct 2.3.4 Community designs ↑ Direct OUTPUTS Innovators 3.1.1 SMEs introducing product or process innovations ↑ Direct 3.1.2 SMEs introducing marketing /organizational innovations ↑ Direct Economic effects 3.2.1 Employment in knowledge-intensive activities ↑ Direct 3.2.2 Medium and high-tech product exports ↑ 3.2.3 Knowledge-intensive services exports ↑ 3.2.4 Sales of new to market and new to firm innovations ↑ 3.2.5 License and patent revenues from abroad ↑ Direct * BA: Behavior Additionality, OA: Output Additionality ** AIP measures countries‟ progress towards the EU2020 Innovation Union. 40 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Vinod Goel PSD/Knowledge & Innovation Specialist ECSPF Ekaterina Koryukin PSD/Projects Officer ECSPF Mohini Bhatia Operations Analyst ECSPF Irina Kichigina Legal Counsel LEGEC Michael Gascoyne Financial Management Specialist ECSPS Nicholay Chistyakov Sr. Finance Officer LOA Arben Maho Procurement Specialist ECSPF Maria Vannari Procurement Specialist ECSPF R.A. Mashelkar R&D Specialist External Supervision/ICR Arabela Aprahamian Senior Operations Officer ECSF1 Paulo Correa Lead Economist FIEEI Edward Daoud Consultant GPOBA Paula Genis Operations Officer ECSF2 Vinod K. Goel Consultant EASFP Dubravka Jerman Program Assistant ECCHR Irina L. Kichigina Chief Counsel LEGEM Lamija Marijanovic Financial Management Specialist ECSO3 Mirela Mart Consultant ECSOQ Dragana Pajovic Consultant ECSF1 Claudia M. Pardinas Ocana Senior Counsel LEGAF Kirsten Burghardt Propst Senior Counsel LEGEM Jean-Louis Charles Racine Private Sector Development Specialist ECSF2 Amanda Marie de Oliveira Schneider Program Assistant ECSPF Murat Seker Economist FIEEI Ljiljana Tarade Operations Analyst ECCHR Maria V. Vannari Senior Procurement Specialist OPCPR Cristina Velazco-Weiss Program Assistant MDI Antonia G. Viyachka Procurement Specialist ECSO2 Iwona Warzecha Sr Financial Management Specialist ECSO3 41 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including travel and No. of staff weeks consultant costs) Lending FY03 16 121.80 FY04 14 168.96 FY05 13 135.30 FY06 0.00 FY07 0.00 FY08 0.00 Total: 43 426.06 Supervision/ICR FY03 0.00 FY04 0.00 FY05 0.00 FY06 14 82.13 FY07 26 138.45 FY08 35 146.46 FY09 24 0.00 FY10 FY11 FY12 Total: 99 367.04 42 Annex 5. Beneficiary Survey Results The development impact of the project was assessed through qualitative and quantitative research undertaken by the Zagreb Institute of Economics (ZIE), particularly related to Innovation programs financed by the project. The overall effectiveness and efficiency of the project, as perceived by actual and potential beneficiaries were investigated along two dimensions. First, the effectiveness and efficiency of the program, i.e. the efficacy of the programs to achieve its objectives as perceived by the private sector using project support and, second, the administrative or institutional efficiency, i.e. the efficient management practices in the organizations that implemented the program. Overall assessment shows that project supported programs received very favorably: (i) The evidence suggests the project had a positive impact on the Croat firms in terms additionalities in terms of technological effort and R&D spending, productivity and profitability, competitiveness (export intensity), employment generation. The program achieved to a large extent the intended outcomes in the Croatia‟s economy. Technological activities are expected to have long term impact on firms‟ performance. Since the program was just completed, only its short term impact was evaluated, and, in this sense, its real impact will extend over the long term only. (ii) The conclusions regarding the efficiency of the management practices of the implementing agencies, particularly BICRO and UKF: a. RAZUM program. All companies that applied to RAZUM program rated it highly on most features. However, they pointed out weaknesses of RAZUM program that need to be improved. The procedure of project application should be simplified, and the evaluation process should be faster. Companies also believe that Croatian enterprises in general are not sufficiently acquainted with RAZUM program, signaling the need for stronger promotion in order to increase the awareness of the program. This finding was confirmed in the survey of potential users (high tech SMEs). The results of the survey show that companies that are not familiar with RAZUM rate the benefits of the program lower that RAZUM applicants. This result indicates that RAZUM could increase the number of applications if companies were better informed about possibility to realize benefits (for example from case studies of successful program graduates). b. SPREAD. Beneficiaries had a positive opinion about the program, with most of them claiming that would definitely or probably re apply with other projects. The general attitude was that financial support received was very beneficial to them. Companies also expressed positive attitude towards SPREAD‟s procedures of evaluation, application and granting the funds. However, they indicated a low level of acquaintance with the program, indicating the need for improvements in communication strategy for SPREAD. Generally, co-financing principle was judged as acceptable for entrepreneurs although some changes would be welcomed. Some entrepreneurs considered that their contribution should have been lower, and others considered that it should have been “in-kind�. The lack of awareness of SPREAD was confirmed in the survey of potential users (high tech SMEs). The results of the survey showed that companies that were not familiar with SPREAD rated the benefits of the program lower that SPREAD applicants. While this testifies to the quality of the program, it 43 also suggests that raising awareness about possibility to realize benefits would increase the number of applications to SPREAD. c. UKF: Researchers were satisfied with UKF program, and 95 percent of them would probably or definitely apply again to UKF with another similar project. They rated UKF program high on all its features. However, researchers considered that the Croatian community was not as well acquainted with UKF program, and thus more effort should be made to promote UKF program in the Croatian academic community. Furthermore researchers pointed out that UKF should invest more money in the research. Since the largest numbers of UKF projects were directed at basic research, interest for commercialization of UKF research results was rather low. While the cooperation with academic community was excellent, the cooperation between researchers and business community was not so successful. The scientists that applied but had not received the UKF funding showed not significant disappointment about the negative outcome: more than a half of them would apply again to the UKF program. Those that would not apply again expressed their doubts about the fairness of evaluation process and the choice of referees. A significant share of all surveyed researchers considered difficult to meet the co-financing principle, especially from the business sector, and/or proposed that the required contribution be lower. UKF non- recipients agreed that the UKF application procedure was well explained and that Croatian scientists were well acquainted with the UKF program, but they had less positive opinion of the transparency of the UKF program, evaluation process and feedback received from the evaluators. Respondents seemed to be relatively confident about the quality of their proposals: they plan to continue working on them and turn to other sources of funding. They admitted that their ability to carry out the collaboration with the chosen foreign institution could be damaged to some degree, but not too much. However, most affected could be the young researchers and PhD students. 44 Annex 6. Stakeholder Workshop Report and Results NA 45 Annex 7. Borrower's ICR and/or Comments on Draft ICR 1. Project background and development The Science and Technology Project (STP), financed through a World Bank (WB) Loan of 30 million EUR, with counterpart funding from the state budget of the Republic of Croatia in the amount of 5.4 million EUR, was designed in line with the technology and innovation supporting program of the Republic of Croatia. The Project beneficiary was the Ministry of Science, Education and Sports (MSES). The primary objective of the Project was providing support to companies based on knowledge and innovations and restructuring selected Croatian Research and Development institutes (RDIs). The second goal had been shifted (after a Mid-term Review) to supporting technology transfer and commercialization of innovation. The end beneficiaries were the Project Agencies (PAs): (a) Brodarski Institute (BI), Rudjer Boskovic Institute (RBI)/Rudjer Innovations (RI) and Business-Innovation Centre of Croatia (BICRO) which received a combination of grants/soft loans from the Loan proceeds, according to pre-agreed formulas and (b) Technology Transfer Offices (TTO‟s) of the University of Zagreb and University of Rijeka, and Unity Through Knowledge Fund (UKF) (within MSES) which received grants from the Loan proceeds. Ultimately, the final beneficiaries were also knowledge-intensive SMEs, researchers and research groups eligible to benefit from sub-financing and innovation financing programs under BICRO, UKF and RBI/RI in accordance with their agreed conditions. The Project, due to its complexity and some external factors (consensus on strategic issues, the government change, etc), went through an somewhat longer preparation period. From as early as 2003, funding under the Project TAL 2 in the amount of 1.6 million USD was utilized, primarily for initial consulting services ranging from studies on the commercial potential of departments at BI funding of two pilot R&D projects towards commercialization at RBI, study tours and education of the PAs management staff, an attempt at mapping business processes at the two institutes, etc. In May 2005 the final Project Implementation Plan (PIP) was drafted. According to the PIP, the initially planned STP implementation period was from November 2005 – November 2009. The expected effectiveness date was set at October 31, 2005, and WB funding agreed in the amount of 31 million EUR. The initial objectives were set as: i) Restructuring of RDI institutions; ii) BICRO component – upgrading technological capabilities of enterprises; iii) implementing the UKF. The Loan Agreement was signed on September 16, 2005, and ratified in the Croatian Parliament in December 2005. However, the effectiveness date was moved to May 11, 2006. This initial delay was due to a combination of unresolved issues: the Loan ratification took longer than expected, due to the various legal procedures, and, most importantly, a crucial document to be adopted by the Government, the Science and Technology Policy, could not be finalized earlier due to a prolonged public debate and consensus building involving all stakeholders in Croatian research and innovation system. The main STP components were: I. Restructuring of RDIs (8.2 million EUR from the Loan). This component includes institution building of selected RDIs (Brodarski institute, Ruder Boskovic Institute and others) through assistance in corporate management, development of business processes and 46 human resources, upgrading of infrastructure, and commercialization of research projects and technology transfer. Technology Transfer Offices (TTOs) from two public Universities, the University of Zagreb (UoZ) and University of Rijeka (UoR) joined the project during implementation. II. The Business Innovation Centre of Croatia (BICRO) component (15,8 million EUR from the Loan) supported several programs conducted by BICRO, a state owned company, whose role is focused on financing technology development in SMEs. These programs were: RAZUM - development of innovative technologies in SME‟s), TEHCRO - upgrading technology infrastructure, encompassing incubators and R&D centers, and IRCRO (SPREAD) - providing grants for financing collaborative research projects between SMEs and public sector RDIs. III. The Unity through Knowledge Fund (UKF) (4.5 million EUR from the Loan) is a facility to attract Croatian scientists and researchers living abroad. The program financed collaborative projects of domestic and diaspora scientists, visits of eminent Croatian scientists and establishing cooperations between domestic and foreign RDIs. IV. Other activities - (1.5 million EUR from the Loan) for support and building of the National Innovation System (NIS) and project implementation. The two initial RDIs participating under 1st component were: RBI and BI. Following prolonged negotiations and the fact that at RBI the research topics are predominantly in basic science (life sciences, physics and chemistry), and at BI‟s research is within engineering and shipbuilding technology, it has been agreed the following: - RBI, as public research institute, mainly funded by the state budget, is going to found a daughter company, Rudjer Innovations (RI), under RBI ownership as the direct participant in the STP, acting as the Technology Transfer Office of RBI; - the BI, as state-owned corporation, re-structuring agenda was agreed, with precondition of stable state subsidy. A Mid-term review of the Project conducted during 2008, led to the Project‟s restructuring which was approved in July 2009 by the WB Board. The Project‟s focus was shifted from the restructuring of RDIs to the objectives of enabling the commercialization of public research and fostering private sector R&D. These aims reflected more accurately what could be achieved with the available funds and given the prevailing institutional conditions and legal framework for in the country. It was agreed on that the precondition for further progress towards the objective of restructuring public RDIs was legislation change, in order to impose better governance within RDI‟s and in order to implement the output-based funding. On the other side results achieved in commercialization of publicly funded research and the generated spillover effects surpassed original expectations. The Project implementation was extended until May 31, 2011, due to accumulated delays in the beginning of the Project, the restructuring undertaken at Mid-term, and to allow for capacity growth of the later-included PAs, which were non-existent at Project outset. 47 2. Project Management The overall responsibility for implementation and coordination of the Project was delegated by the Government of Croatia to the MSES. The Project management structure included a professional independent Project Management Unit (PMU) responsible for day-to-day project management on behalf of the MSES, as defined in a formal Delegation of Authority. These responsibilities included project management, procurement, disbursement, financial management, and monitoring and evaluation (M&E) for the entire project. The PMU reported to a State Secretary (later Director of the Science Directorate), nominated as Project Coordinator. The PMU consisted of a Director, Financial Expert, and Procurement Expert (initially engaged at 50% work time, later two full-time consultants as the volume of procurement grew). A Project Council, consisting of the Project Coordinator, PMU, Directors of the PAs and a representative of the MoF, was held annually when project-wide issues were discussed. Issues were also discussed with all stakeholders at the annual Joint Portfolio Reviews (JPRs) held at the MoF. On the WB side, the Task Team was led by a Task Team Leader (TTL), with permanent support from finance and procurement specialists. Other experts were consulted on an as-needed basis (mostly concerning RDI issues, and later focusing on M&E). Mutual obligations and responsibilities were setup through a three-tier set of legal documents: following the Loan Agreement, Project Agreements (PAs) were signed between the WB and PAs (BI and BICRO), and Subsidiary Financial Agreements (SFAs) regulated financial obligations of the PAs (BI, RBI and BICRO) towards the Government (represented by both the MoF and MSES). The PAs were given responsibility for implementation of the activities related to their components. The implementation was organized on the basis of annual Business Plans, which required prior approval by both MSES and WB. The UKF Program, as a fund with no legal entity to support it, was operated by consultants directly from the MSES with logistic support from BICRO. The comprehensive set of project documents created prior to effectiveness, including the Project Appraisal Document (PAD) and Project Implementation Plan (PIP) provided a general framework for project implementation, defining Project Management responsibilities (as outlined above), costs and financing, procurement and financial management arrangements, M&E procedures, down to pre-agreed formats for reporting and drafting Terms of Reference (TORs) for tenders. The Operational Manual was revised during implementation, whereby the PMU/MSES and TT/WB jointly addressed procedural issues and provided guidelines for them. The funds-flow agreement included an understanding on the usage of both Bank funds and counterpart funding from the State budget. A Special account was opened for the duration of the project in a commercial bank, and managed by the PMU. The PAs were provided funding according to annual procurement plans, which passed approval both from the MSES and WB. Procurement for the Project was carried out in accordance with the WB‟s “Guidelines: Procurement under IBRD Loans and IDA Credits�, and “Guidelines: Selection and Employment of Consultants by WB Borrowers�. Guidelines for Sub-Financing (SF) and Innovation Financing (IF) categories were set in the Operational Manual. Supervision of the goods and services under SF/IF was delegated to BICRO/UKF/RI, respectively, as the implementing agencies. 48 Overall responsibility for financial management issues was assigned to the PMU, whose fiduciary responsibilities included ensuring that project activities were carried out in accordance with agreed project objectives and performance targets. Disbursement was based on replenishment of the Special Account or occasional direct payments. A Financial Management System (software) was procured at the outset, and utilized throughout the Project life. FMRs were issued quarterly as well as the reports on used STP funds for each PA. Financial statements of the Project, as well as individual PAs, were audited annually by an Independent Auditor, approved by the WB. STP and individual PAs were also occasionally supervised by the MSES internal audit, and MoF. Regular procurement and financial supervisions were conducted by WB procurement and financial experts and the State Audit. M&E arrangements were outlined in the PIP, and an early set of Key Performance Indicators (KPIs) was established. During Project implementation the KPI monitoring was significantly enhanced, following expert input and empirical knowledge gained. 3. Key Stakeholders The Ministry of Science, Education and Sports (MSES) acted as the main project beneficiary and had overall responsibility for implementation of the STP. The MSES is also the central authority in charge of implementing the National Innovation System (NIS), comprising activities and instruments for fostering the development of Croatia as a knowledge-intensive economy, based on innovation. The NIS supports whole innovation cycle, from basic research, technology transfer from RDI‟s to the private sector, stimulating innovations in the private sector, and improved positioning of the national economy in view of the expected EU-accession in the near future. The MSES provided direct support for the Unity through Knowledge Fund (UKF). This program was aimed at: (i) encouraging Croatian scientists and professionals working abroad (Diaspora) to connect with local scientists; (ii) fostering Croatian institutions and researchers to utilize potentials of the Croatian scientific and professional Diaspora. UKF‟s mission remains to unify the scientific and professional potentials in Croatia and the Diaspora for the development of a society based on knowledge. In order to achieve this task, UKF set the following goals: fostering internationally competitive research, research that provides added values to the Croatian economy and supporting the projects that develop research infrastructure. The projects financed by UKF were chosen on a competitive basis through an unbiased peer-review process based on excellence. Rudjer Boskovic Institute (RBI) is the largest Croatian research institute, owned by the Government of Croatia. The fields of activities of the divisions are primarily in basic and applied research in natural sciences (physics, chemistry, and biology), but also in oceanography, biomedicine, genetics and computing. RBI was founded in 1950 in Zagreb, as a center for advanced scientific research, with oceanographic facilities in Rovinj. From its inception, RBI has been dedicated to fundamental research. In parallel, RBI has engaged in a limited amount of applied research, jointly with industrial companies, both within Croatia and internationally, collaborating with other institutes and industries, and cooperating with governmental institutions 49 related to national defense, health care, food production and environmental protection. Additionally, RBI offers expert advice and serves as a quality control reference for some products, standards and measures. RBI has a well-organized infrastructure, which includes a computing center, animal facilities and a large number of valuable experimental devices (Tandem Van der Graaff, EPR, X-ray and NMR, core facility for molecular genetics, MS, IR, Raman spectrometers, a cyclotrone, PET-camera etc), which were further enhanced through procurement under STP. The Institute has over 900 employees with over 500 PhD researchers. In terms of high science and innovative capacity, RBI is a national asset. RBI could contribute tremendously towards elevating the scientific and technological content of Croatian industry by focusing its scientific talent to creating technological advancement, while maintaining its scientific excellence. A preliminary review of RBI by international scientific experts confirmed that the quality of research across a range of areas has been maintained at a high level and outputs in these fields have the potential for technological breakthroughs and commercial applications. The Agency for Innovation and Technology Transfer of the RBI, Rudjer Innovations (RI), was registered as a limited-liability company in 2007, for the purpose of protection and placement of the know-how, innovations, patents, technology transfer, incorporation of spin-off and joint- venture companies and connecting RBI with other scientific, technological or industrial entities at home and abroad, aiming at production of new technological high-value products that are competitive in the world market. The mission of RI is to be the main facilitator in connecting scientific institutions in Croatia (primarily RBI) with financial and investment entities at home and abroad. Brodarski Institute (BI) was founded in 1948 as an R&D institution of special interest to the Republic of Croatia in the field of marine industry, science and technical systems. Between 1991 and 1993 BI was transformed from a budgetary financed institution into a limited liability company fully owned by the Government. Since then BI has been financed from commercial contracts with both the public and private sector. During the 90-ies BI significantly reduced operating costs as well as non-marketable activities. Knowledge acquired in defense related work was directed towards development of R&D projects for the needs of the national market. Over this period the number of employees was reduced from 450 (in 1991) to 160. Among the 7 public universities in Croatia, the University of Zagreb (UoZ) is the largest in every aspect. With its 63,000 students studying for Bachelor and Masters Degrees, as well as 2,000 PhD students, 5,500 members of academic staff, and 8,000 employees distributed in 29 faculties, 3 academies and 1 University center. University of Zagreb, as the backbone of Croatia research system, in the field of natural sciences, biomedicine and several areas of engineering and applied biosciences provide significant potential for innovation and technology transfer. The establishment of the UoZ Technology Transfer Office (UoZ TTO) in 2008 was the first necessary step towards improving IP protection and technology transfer. The TTO was created as a university infrastructure that serves to promote development and commercialization of innovations and discoveries resulting from scientific research conducted at the UoZ. 50 The University of Rijeka (UoR) is situated in the coastal industrial city of Rijeka, with several faculties located in neighboring towns. Like other Croatian public universities, it is a state- funded institution. With its 17000 students, 1000 researchers, 250 PhD students, 350 academic staff, and a total of 1600 employees, it provides a sound research base. The University of Rijeka Technology Transfer Office (UoR TTO) was founded in 2008. Its mission is to extract fair market value of the University's intellectual property and stimulate creation of future commercially viable IP, using best business practices for the benefit of the University and the community. The Business Innovation Centre of Croatia (BICRO) is an innovation and investment limited liability company established by the Croatian Government in 1998, with the goal to facilitate technology transfer and commercialization activities primarily in the sector of technology driven small and medium-sized (SMEs) companies, contribute to the creation and development of a private equity industry, and promote the establishment and development of science and technology parks/centers and related institutions. The primary objective of BICRO is to promote the competitiveness of Croatian industry and the service sector by assisting in the creation of world-class technology and technological know-how. This objective is pursued through the following programs: The RAZUM Program (Development of knowledge-based companies) is a seed capital program and its focus is on co-financing development costs of new products. It offers financial support to technologically advanced firms based on knowledge, where the level of technological innovation, quality of management, commercial potential, competitive advantages and quality of the Business plan are assessed through a transparent pre-determined procedure. The Research and Development Program (IRCRO) aims at encouraging SMEs to engage in R&D. The program envisages utilization of extensive facilities available within the universities and other independent technology institutions in the country. Each project under the IRCRO Program involves cooperation between an industrial firm and an RDI. The resulting project is carried out together with the RDI and is jointly funded by BICRO and the company, through a matching-grant scheme. The Technology Infrastructure Development (TehCRO) program is focused on developing efficient and self-sustaining technology transfer facilities and business support infrastructure to meet the needs of private entrepreneurs, as well as researchers and academia. The Program provides 3-5 years financing to projects aimed at the establishment, upgrading and/or development of Technology Business Centers, Technology Incubators and Science & Technology Parks. Researchers from Croatia's universities and research institutes, as well as entrepreneurs, can receive funding to turn their ideas into a global business with the help of the Proof of Concept Grant Fund (PoC). This Fund is administered by the network located at the Universities of Zagreb, Split, Rijeka and Osijek, as well as in Varaždin and Čakovec techno-parks, with BICRO responsible for the selection process and financing. 4. Assessment of operation’s objective, design, implementation and operational experience The role of the STP in last several years‟ development of Croatian R&D system has been very significant. Generally, the STP acted as an important catalyzer for structuring and improvement 51 of the Croatian national innovation system, as a precondition for increasing research and development (R&D) activities, as a basis for high technology and innovation-based economic development, both in private and public sector. It helped creation of an infrastructure for increasing the level and quality of commercialization processes at universities, institutes and private companies. The STP programs triggered investments from EU pre-accession programs in R&D sector.; improvement of technology transfer process and intellectual property protection on research institutions and increasing of awareness of the importance of research commercialization for economic development. The STP acted as a valuable and complementary instrument for implementation of national research and technology policy. The Project has proved itself as an essential facility over the last 5 years for strengthening of included R&D institutions and SME‟s who received financing as end beneficiaries, their increased orientation towards commercialization of research. However, the PAs participating were very diverse, because of the various starting position and framework of each PA: RBI as the largest public RDI in the country, BI as a company of applied technology orientation surviving on the market with little public funding, BICRO - a newly formed state agency for financing technology-oriented SMEs, the Unity through Knowledge Fund created as a Ministry‟s program for supporting excellent researchers‟ international cooperation, and the newly formed TTOs at the University of Zagreb and at the University of Rijeka, which joined during the implementation. The Project was heavily dealing with institution building, with raising awareness of the importance of intellectual property protection and knowledge and technology transfer to the private sector. As previously explained, the revised Development Objective (DO) was to enable RDIs to commercialize research outputs, as well as to increase the ability of enterprises, particularly SMEs, to invest in R&D activities. The need for revision of DO indicates that initial goals have been set not taking into account boundaries of overall legal framework of Croatian public sector. The built-in flexibility allowed for Project restructuring during implementation: based on lessons learned some goals were changed, restructuring of public institutions was abandoned, and more emphasis was placed on technology transfer and commercialization of knowledge. Some components that could not be launched due to objective circumstances were cancelled (the Venture Capital Fund prepared by BICRO, due to lack of available funding coinciding with the global crisis reflecting on Croatian economic and fiscal situation). The WB was justified in postponing the Effectiveness date pending on the adoption of the Science and Technology Policy for the Republic of Croatia 2006-2010 (S&T Policy). The document provided a valuable framework for the implementation of the Project. The S&T Policy was a strategic document stating goals and instruments in the mid-term period. The S&T Policy's overall aim was to stimulate scientific excellence and enable the transfer of knowledge and results of scientific discoveries to industry and business, in order to increase competitiveness and generate sustainable growth and productivity. The document listed a set of specific objectives for promoting business innovation and technology development. It included a brief 52 analysis of how the objectives were to be met, including a Policy Matrix where policy measures aimed at encouraging science and industry cooperation were related to strategic objectives, critical challenges, and target groups, followed by policy measures linked to groups responsible for their implementation. The Action Plan: Science and Technology Policy of the Republic of Croatia 2007-2010 (Action Plan), was adopted by the Croatian Government on April 18, 2008. The Action Plan built on a basic gap analysis of the prevailing NIS challenges, set progress indicators for monitoring R&D supply and absorption capacity and demand, and listed priority actions for individual Policy elements including: improving the legislative framework, educating the S&T community towards commercial exploitation of R&D, improving financing of the S&T sector, facilitation of innovation-based start-ups, integration with the EU and international environment and intensifying R&D in industry. The biggest lesson learned during Project implementation was that it is easier to undertake institution building, rather than reforming existing institutions, due to the limited timeframe of implementation. The key factor for success remain high-skilled people: dedicated experienced professionals with appropriate additional training. The ultimate factor underlying implementation success remains the prolonged preparation, governance setup and overall design of this very complex Project. The described setup and acquired implementation experience enabled total disbursement above 99% of the available funds, in line with the Project development objectives, and under strict enforcement of all applicable operational procedures. 5. Assessment of the outcome of the implementation against agreed objectives The most important outcomes achieved during STP implementation are as follows: Rudjer Innovations, the commercialization agency of RBI, concluded 12 licensing agreements valued at over 750,000 EUR, with customers including prestigious institutions such as MIT and Brown University; five patents were granted and 37 patent applications were filed. Five spin-off (science-based) companies were launched (eight planned) surpassing the target set for market value generated, of which two are already self-sustainable and one (RMC) promising even higher returns in the near future. Several other potential spin-offs are under development in the pre- commercial phase. Despite numerous activities aimed at enhancing the commercialization of research, including adoption of an IP Policy, the industry/business sector is still not fully aware of the existing innovation potential at RBI, leaving space for future improvement. Brodarski Institute concluded 41 research and service contracts with industry valued at 9 MEUR, and increased its share of revenue from the private sector to 45% at project closure (compared to a baseline of 22% in 2006). STP funds were utilized to enhance facilities with proven market potential and upgrade infrastructure in the core hydrodynamics laboratories. BICRO directly supported 89 SMEs qualified for its programs through the rigorous application process. The STP, specifically, through its component of supporting R&D in private companies, has helped to restructuring and developing Croatian innovation agency – BICRO. The RAZUM program for development of knowledge-based companies approved 31 projects for funding (out 53 of 135 applications received) and contracted 22 projects, with a total value of 21.9 million Euros - 66% of that financed by the Project funds and 34% directly contributed by the private sector. Those companies employ a total of 576 people, and their total income in 2010 was 33.83 million Euros. SPREAD/IRCRO, program for supporting cooperation between SME‟s and public research institutions financed in total 17 cooperative projects and attracted 1.35 EUR of private funding per 1 EUR of public funding provided. The Proof-of-Innovative-Concept program supported 46 projects with a total value of 1.86 million Euros - 34% is contributed by the private sector. The projects generated 29 patent applications, 19 technical feasibility demonstrations and 40 prototypes. Finally, the technology infrastructure development program TEHCRO financed totally 12 projects, with a total value of 8.93 million EUR, where 1 EUR invested attracted 1.43 EUR from other funding sources. In 2010 there were a total of 52 high-tech companies housed within 6 incubator centers, employing a total of 494 people, and 217 new jobs were created in tenant companies between the years 2008 – 2010. Participation of BICRO in the STP was a key reason that BICRO is successfully benefited from the IPA-funded project (16 million EUR) of building a new biotechnology incubator BioCentar. While difficult to single-out any particular success story, the MariBIC Center of Excellence for mariculture providing services to the traditional Ston bay fish and shellfish hatcheries promises interesting developments, both due to its unique position and natural resources, solid scientific base and prudent oversight provided by BICRO. The UKF facility financed 91 projects (target: 50), of which 59 successfully concluded by STP closing date. Of these, 28 contracts were with industry estimated at 2.7MEUR, and attracting further investments of 650 KEUR from the partners. The cooperative projects with foreign RDIs attracted 1.1 MEUR of additional investments. The target set for the number and value of contracts was easily exceeded. 544 researchers participating in the UKF program from over 100 public research organizations (70 from abroad) published 180 articles in the most prestigious international scientific journals. Another unpredicted spillover effect was the high success-rate of the UKF projects which went on to apply for highly competitive 7th Framework Programme funding (30% compared to the national average of 16%), bringing additional funding valued at 6.9 MEUR to domestic researchers. The historical lack of „patenting culture‟ and reluctance to academic entrepreneurship among university researchers was challenged by the establishment of the University of Zagreb TTO in 2008. The University Senate accepted in 2008 an IPR Policy, which was the first of its kind in Croatia, denoting a significant paradigm shift among University researchers. The UoZ TTO achieved 8 patent filings and 18 IP transfers within its 2 years involvement on the Project. More significantly, it set a standard of highly trained individuals who were quick to embark on EU- funded projects, thus creating a realistic basis for future self-sustainability. Due to its late start in the STP, the University of Rijeka achieved limited results. However, the IP protection culture has been implanted, and IP Policy and Rulebook adopted by the University Senate in 2010. Altogether, within the STP, three major Croatian academic institutions established the fully functional technology transfer offices (Institute Rudjer Boskovic, University of Zagreb and 54 University of Rijeka), and introduced best practices in technology transfer previously missing in Croatia. All three institutions, because of the experiences achieved through implementation of the Project, were successful in obtaining projects for further development of technology transfer from the Science and Innovation Investment Fund (near 1 million Euros), which is activity under IPA Regional Development Program. At the end of STP the Zagreb Institute of Economics made an independent evaluation of the impact of RAZUM, IRCRO and UKF programs, supported under STP. Among others, the assessment showed that the STP programs RAZUM and SPREAD have enabled companies to invest more in R&D, and helped them carry out more innovative projects and in a shorter period of time (the “additionality,� as opposed to “full substitution� effect). From 2006 to 2008, the share of Croatian SMEs that have new or significantly improved products new to the market raised from 7.2 percent to 10.8 percent (EU Community Innovation Survey). 6. Evaluation of the Borrower’s and Bank’s performance The STP was primarily an institution-building experience, resulting in the development of the National Innovation System. Looking back after Project completion on May 31, 2011, it is safe to say that the STP has profoundly influenced all the involved institutions, achieving unprecedented institution-building and capacity growth, most visibly highlighted in BICRO‟s achievements. Setting up RI and the TTO at the University of Zagreb has changed the way technology transfer is conducted in Croatian research institutions, and the UKF operation have set-up new standard for transparent competitive grant schemes in the R&D sector. Moreover, the STP influenced numerous SME partners from the private sector (confirmed by an in-depth study performed towards the end of the Project by the Economics Institute), and has set managerial and financial-control standards which remain a firm reference for future projects. The PAs supported form the core of the NIS, whose framework was defined and enhanced during the Project implementation, making it the most advanced in the region. The MSES as the Project‟s initiator and implementation entity had the vision of Project‟s goals which had to be achieved. The complexity of the Project involving many different stakeholders / project agencies (institutes, companies, entities within universities and the Ministry) was a big challenge and required intensive coordination and cooperation. The Project Management Unit (PMU) was the right solution to over-bridge numerous expectations from the Project Agencies and to steer them, with the MSES‟s support, towards the Project‟s goals. The STP Steering Committee significantly contributed to identifying the problems, finding solutions and focusing to the main goals. However, there is always space for improvements in the area of Project Agencies‟ understanding WB procedures, efficient responses to the MSES‟s, WB‟s and PMU‟s requests and requirements, getting all entities informed on relevant issues within PA‟s etc. The World Bank had very important and positive role as partner, controller and advisor in the Project implementation. It contributed in project designing, budgeting, defining the measurable project indicators, setting proper procedures etc. WB shared international experiences in the similar projects, intermediate with the relevant international institutions and provided relevant resources regarding bench marking. There is still room for improvement the efficiency of response to the problem resolution in the everyday running of the Project, because of its 55 complexity and in the communication with Project Agencies. As suggested by the Bank, the Project allowed certain autonomy of the Project Agencies. However, this left insufficient space for later interventions of MSES as the actual responsible body. 7. Proposed arrangements for future operation of the Project Agencies The critical issue following successful implementation of STP remains the lack of available funding within the country, due to fiscal constraints, for stimulating research, development and innovation commercialization across the public and private sectors. Currently only 0.7% of the GDP is invested in R&D (compared to 1.2% several years ago when the Project was starting. Since EU-accession can now realistically be planned at mid-2013, the biggest short-term danger lies in a lack of funding over the following 2-3 year period for the NIS and its key institutions. Besides setting up National Innovation Policy, legislative customization in science and research, R&D tax incentives and other initiatives, the Croatian Government is strongly committed to continue the positive trends in innovation and technology transfer development by promoting usage of EU pre-accession funds. R&D Policy is one of the priorities in the Government Economic Recovery Program. STP also contributed to increase the capacity of Croatia‟s institutions to absorb EU pre-accession funds and prepare for a better use of EU structural funds in the future. MSES has designed IPA- financed Science and Innovation Investment Fund (6 MEUR) for technology transfer based on STP model. BICRO was awarded 18 MEUR from EU IPA (Instrument for Pre-Accession Assistance) for a Biotechnology Incubation Center (project preparation funded by STP). By increasing the participation of local researchers in FP7, UKF program contributed to further integrating Croatia‟s research to the European Research Area (ERA). There have been extensive contacts between the MSES and MoF from the Croatian Government side and high-level WB officials, regarding the launch of a “repeater project� which could smooth the transition and prepare key institutions within the Croatian NIS for absorption of EU funding. RBI, in partnership with the University of Rijeka, is currently participating in an IPA project to promote continuing education in the area of IP, stronger support in evaluation of IP and its protection, as well as providing support in the process of negotiating contracts with industry/business, creating a database of scientific infrastructure and expertise, in order to provide both RBI researchers and partners from the industrial/business sector better insight into the potential in scientific research, measurements and other available services. The sustainability of RI and is functions became unquestionable by RBI management, which is currently implementing institutional restructuring process of whole institution, in order to reach higher competitiveness of research performance and technology transfer. Projects and services in the marine technology sector are influenced by the global crisis and the situation is the same in the shipbuilding industry. BI has recognized this volatility and has successfully switched marketing and sales activities to international markets. Projects in the 56 advanced technologies sector show promise, especially expertise gained in Municipal Waste Management. The BI group for EU projects is currently engaged in four contracted FP7 projects (with leader status in two of them). The ultimate goal is to transform BI into the leading Croatian RTO organization in the field of advanced technology and applied sciences, thus setting the foundation for long-term sustainability. BICRO has undertaken organizational changes converting its legal status to a Government Agency in 2010 and in parallel assumed a role in programming and implementation of future EU structural funds in the area of innovation and technology, on behalf of the MSES. However, EU funds shall become available in 2013, leaving uncertainty in funding for the transition period. Through the involvement in Pre, BICRO secured an 18 MEUR investment for establishing a biosciences innovation and commercialization center (BIOCenter), as an important part of the research and business infrastructure for the build-up of the biotechnology sector in Croatia, from the IPA fund. The UoZ TTO has achieved significant results during its short lifespan. A strong procedural framework has been established for IPR. A framework for knowledge exchange and spinout is still under development and making strong progress. The fund raising and support to University management function has been successfully developed. Currently funding is available from the TEMPUS OPUS project, and under 2 IPA/SIIF projects; the latter is financing technology and expertise mapping. A strategic roadmap discussing short-to mid-long term activities concerning sustainability has been developed. UKF sustainability was considered high-risk from the outset, due to the ad-hoc institutional setup as a MSES program. But UKF undisputed success makes the good foundation for securing future funding from budgetary resources and FP7 projects. 57 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders NA Feedback has not been requested from any other stakeholders. There were no co-financiers or partners in the project. 58 Annex 9. List of Supporting Documents World Bank documentation:  Country Assistance Strategy, May 11, 1999 (Report No. 19280 HR)  Country Assistance Strategy Progress Report, September 28, 2001 (Report No. 22633 HR)  Country Assistance Strategy, November 2004 (Report No. 30717 HR)  Country Partnership Strategy FY09-FY12, August 13, 2008 (Report No.44879-HR)  Country Partnership Strategy Progress Report FY09-FY12, August 22, 2011 (Report Number: 61715-HR)  Project Concept Note, 2002  Aide Memoires ( 2003-2011)  Back-to- office reports and Letters to Government (2002-2011)  Implementation Status Reports (2005-2011)  Project Appraisal Document (Report No. 25748)  Signed Loan agreement and Project Agreements  Project Restructuring Papers  Amendments to Loan Agreement  Joint Portfolio Review Reports  Project Management Reports Other project related documents:  Croatian Program for Innovative Technological Development (HITRA), April 2001  Croatia in the Twenty-First Century-Science (OG 108/2003)  Science and Education Act, (OG 123/03)  Science & Technology Policy of the Republic of Croatia 2006 − 2010, 2006  Action plan 2007-2010,  Action Plan for Increasing the Number of Investments in Science and Research, 2008  EU Screening Report-Croatia, Chapter 25- Science and Research, February 2006  Communication from the Commission to the European Parliament and Council - Enlargement Strategy and Challenges for 2011-2012, COM(2011) 666-final  Commission‟s opinion on Croatia‟s Accession in EU, COM(2011) 667  Commission Staff Working Paper, Croatia 2011 Progress Report, SEC  Innovation Union Score Board 2010  Europe 2020 Flagship Innovation Union, SEC(2010) 1161 59 IBRD 33394R1 CR O ATI A This map was produced by the Map Design Unit of The World Bank. The boundaries, SELECTED CITIES AND TOWNS MAIN ROADS colors, denominations and any other information shown COUNTY (ZUPANIJA) CAPITALS RAILROADS on this map do not imply, on the part of The World Bank NATIONAL CAPITAL COUNTY (ZUPANIJA) BOUNDARIES Group, any judgment on the legal status of any territory, or any endorsement or RIVERS INTERNATIONAL BOUNDARIES acceptance of such boundaries. 14°E 15°E 16°E 17°E 18°E 19°E A US TR U AU STR I A To Zalaegerszeg To MEDIMURSKA To Graz Kaposvar Cakovec To Varazdin VARAZDINSKA KOPRIVNICKO- HUNGA RY Ljubljana Koprivnica KRIZEVACKA Krapina KRAPINSKO- Durdevac To 46°N Baja 46°N ZAGORSKA To SLOVEN SLOVENI S LOVEN I A To ZAGREBACKA Bjelovar Pecs To Pecs To Ljubljana ZAGREB Virovitica Sombor Ivanic BJELOVARSKO- ZAGREBACKA GRAD Grad BILOGORSKA OSJECKO- VIROVITICKO- To To To ZAGREB Daruvar PODRAVSKA BARANJSKA To Sombor Trieste Trieste Ljubljana Osijek K up a Sisak Karlovac Sav POZESKO-SLAVONSKA Nasice a Novska ˘ Pozega Vukovar un D ISTARSKA Rijeka ay ( Glina Danube) Pazin Vinkovci SISACKO-MOSLAVACKA BRODSKO- Slav ˘ Porec To PRIMORSKO- KARLOVACKA POSAVSKA Brod VUKOVARSKO- Novi Sad GORANSKA SRIJEMSKA To Dvor To To To Stalije Krk Bihac Prijedor Banja Luka Ruma 45°N 45°N Senj To Doboj To To Pula Doboj Tuzla Otocac To Bihac Cres Rab LICKO- Jablanac SENJSKA Un a Losinj Karlobag Gospic Pag Udbina SERBIA Gracac To BOSNI A A ND BO SNIA AND Zadar ZADARSKA Glamoc HERZEGO VINA Pasman 44°N Dugi 44°N Knin Dinara Otok (1830 m) Vodice To Sibenik Livno SIBENSKO- Sinj Adr iat ic KNINSKA Trogir Split Sea Imotski To Mostar SPLITSKO- I TA LY DALMATINSKA Brac Makarska To Mostar Hvar 43°N Vis Ploce Metkovic 43°N Korcula Peljesac DUBROVACKO- MONTENEGRO Mljet To NERETVANSKA Trebinje Dubrovnik CROATIA Gruda To Podgorica 0 20 40 60 Kilometers 0 20 40 Miles 42°N 14°E 15°E 16°E 17°E 18°E 19°E JULY 2006