Document ID: IPF 10-006 VINNOVA Dno: 2009-04589 Overall Report on the Pilot Peer Reviews 21 JUNE 2010
This report was prepared by: Eelco Denekamp Contact details: NL AGENCY (INNOVATION) THE HAGUE, NETHERLANDS Phone: +31 88 602 5265 e-mail: eelco.denekamp@agentschapnl.nl Disclaimer: The views expressed in this report are those of the author. They do not necessarily reflect the opinion or position of the European Commission and in no way commit the organisations involved in the peer reviews. PAGE 2
PAGE 3
Overall Report on the Pilot Peer Reviews 21 JUNE 2010 TABLE OF CONTENTS 1 INTRODUCTION... 5 2 THREE PILOT PEER REVIEWS... 6 3 GOOD PRACTICE ELEMENTS AND LESSONS LEARNED... 7 4 SUGGESTIONS FOR THE PEER REVIEW PROCESS... 9 5 NEXT STEPS... 11 PAGE 4
1 INTRODUCTION Three Pilot Peer Reviews were conducted in the months March, April and May 2010. These reviews are part of the activities of INNO-Partnering Forum (IPF). The INNO- Partnering Forum (IPF) is an INNO-Net. It was established as a project under the auspices of the European Commission (DG Enterprise and Industry) for the period 2009 2012. The aim of IPF is to identify, develop and exploit synergies between public innovation agencies in Europe and propose new approaches to innovation support for SMEs. The project will in particular explore and test new ways of service delivery, aiming to accelerate the take-up of the most advanced innovation mechanisms with proven efficiency and impact. IPF is carried out by a consortium consisting of the following six partners VINNOVA(SE, co-ordinator), Tekes (FI), Technology Strategy Board (UK), Enterprise Ireland (EI), NL Agency (NL) and FFG (AT). Work Package 2 (WP2) of IPF is responsible for the peer reviews (PRs). The work started with three pilot peer reviews. A pilot peer review is a full fledged peer review, at its learning stage. This implies that some of its procedures might be adjusted on the basis of experience. Also the relationship of the peer review to other activities under IPF is under development during these pilot PRs. For the pilot peer reviews the cases (programmes) were provided by the partner organisations and the members of the review teams come exclusively from the partners. PAGE 5
2 THREE PILOT PEER REVIEWS The first pilot peer review was on the Small Business Innovation Research programme (SBIR) in the Netherlands. The PR visit took place the 18th and 19th of March 2010 in The Hague, Netherlands, at the premises of NL Agency. The second pilot peer review was on the Small Business Research Initiative (SBRI) in the UK. The visit took place the 28th and 29th of April 2010 in Swindon, UK, at the premises of the Technology Strategy Board (TSB). The first and second scheme is very similar. Both are based on the US SBIR scheme that stimulates precommercial procurement of R&D. The third pilot peer review was on the Funding for Young Innovative Enterprises (NIY) in Finland. The review visit took place on the 10th and 11th of May 2010 in Helsinki, Finland, at the premises of Tekes. The aim of this third scheme is to generate new innovative high growth enterprises. The three reviews have led to three reports. They are available separately. PAGE 6
3 GOOD PRACTICE ELEMENTS AND LESSONS LEARNED To formulate good practice elements of the three schemes in general is complicated because of the differences between scheme 1 and 2 on the one hand and scheme 3 on the other hand. Therefore we summarise below the main practices and lessons per review. SBIR (NL) The SBIR scheme in its execution is more than just a financial instrument it also includes brokerage, coaching etc. Public Relations and communication has helped to position this instrument. Innovation is placed in a broader context of relevant ideas and relevant problems (SBIR is an example of problem oriented or demand driven innovation policy). SBIR is a contract to deliver, not a subsidy.. Companies often appreciate this. Short time to contract (6-8 weeks after deadline submitting proposal) and simple procedures. Enthusiasm within other ministries (on the basis of personal contacts and networks). SBRI (UK) SBRI is implemented by the TSB as a very flexible scheme, which can have a greater or lesser involvement of the department, depending on the competencies of the department involved. The SBRI process is characterized by light touch administration and simple procedures. There is a short time for the evaluation of proposals. Government agencies come in contact with new companies and new solutions, beyond their existing network and supplier base. For companies it means first of all the possibility to enter into new markets, or to apply new ways into existing markets. SBRI is a way of funding R&D in a new manner: from state aid to procurement. It thus complements state aid funding of R&D. The SBRI scheme means government involvement over a longer period of time ( stay tuned ) and a continuous supplier-customer dialogue. This is especially helpful if application of the new product demands changes in the way the customer is organized. The public sector involvement with innovation is broader than just financial (mere funding of R&D projects), because it also involves regulation aspects and the organisational or institutional dimension of innovation. PAGE 7
NIY (FI) The NIY scheme has a solid budget. From the perspective of an individual firm, there are substantial amounts of money involved against a relatively low administrative burden. Almost all costs which efficiently contribute to the achievement of the business goals are eligible. This makes the scheme very flexible. The use of an integrated multistaged process gives NIY a longer term view. Candidates are exposed to the VC community and get the informed opinion of seasoned VCs on the opportunities of the company for further funding. The program is run with a very small number of administrative staff members. This is due to the fact that it uses the contribution of other Tekes staff and external people (internal industry experts, external VC panel members, Vigo staff) in a rather smart way. The use of the external panel of VC experts is beneficial to both Tekes and the candidates. It gives NIY direct access to the VC community and is an effective way to attract good candidate recipients. The small size of the administration, in combination with the administrative independence, makes NIY very effective and decisive. Decisions on the crucial admission of firms into phase one are made by NIY s director of funding and do not require lengthy negotiation processes. In a similar vein, NIY staff knows all firms personally. They effectively act as account managers to their firms. Because NIY also manages some traditional Tekes R&D grants, the program can act as a one stop shop for small firms. PAGE 8
4 SUGGESTIONS FOR THE PEER REVIEW PROCESS The fact that we are dealing with pilot peer reviews implies that we will develop further the peer review as a method. During and after the three visits, the following observations and suggestions for improvement were made: It is good to have customers (both companies and ministerial departments) and the responsible organisation (support agency) as a discussion partner. As an addition, it would be useful to invite a member of the panel evaluating the proposals. The self-assessment questionnaire completed by the host agency - was useful and should be used both prior to and after the review visit (also in support of the editing process of the review report). Collection of questions of peer review team members before the visit is useful. In the preparation phase we should dig deeper in the context of the measure and the national innovation system to better assess the possibilities for transfer of the measure. The actual assessment of transferability is part of work package 3; The first presentation(s) during the visit should be on the scheme itself. This makes presenations on the context later on better understandable. The wrap up at the end of the visit with the SWOT analysis on a flip-over is a useful way to summarize the remarks and suggestions by the review team; It is important to have sufficient time available for the peer review visit, and in particular for the (preparation of) the closing session. More room for discussion/analysis within the team would be welcome. To insure active involvement of the peer review team members, there should be a close link between the selection and invitation of team members and the expression of interest for specific themes, areas and types of programmes (see IPF Invitation No 1); Active involvement of the peer review members does not only apply to the visit and the review dialogue, it also means investing sufficient time in the preparation phase prior to the visit and in the reporting phase (commenting on the draft version) after the visit. Preferably, interlocutors should be questioned separately to avoid a possibly inhibited discussion. Build in an extra round for remarks and questions to the host agency after the visit of the peer review team. Send (some) questions beforehand to scheduled interlocutors. Ask them for introductory presentation, not just Q&A. On a more practical note: review team members should stay in the same hotel (to stimulate (informal) contacts). PAGE 9
Size of the review team is relevant. Eight people in the review team would be too many. A number of six is about right. The team might occasionally be split up in order to create more time for (more) interviews. It is useful for the team members and certainly for its secretary to receive copies of the presentations made at the review. PAGE 10
5 NEXT STEPS There will be various follow-ups to the set of three pilot peer reviews which were conducted in the period March May 2010. This over-all report together with the three reports covering the pilot peer reviews will be submitted in hard copy to the IPF co-ordinator, formally completing three peer reviews (deliverables). An invitation was published for participation in a next round of peer reviews. Public innovation agencies were asked to express their interest in providing interesting schemes as review cases or in joining review teams. Sixteen applications were received and a selection (in the first instance of the most suitable review cases) will be made. This will be done by the WP2 co-ordinator in conjunction with the IPF co-ordinator and the MT. The next peer reviews will be held in the Fall of 2010. Once the cases have been selected, review teams will be composed and their leaders appointed. Dates of the review visits will be agreed between host agency, team leader and WP2 coordinator. The guidelines which were used for the pilot reviews will be updated this summer by the WP2 co-ordinator on the basis of the experiences so far and suggestions made. A meeting will be held (30th June) with the WP2 and WP3 co-ordinators and other MT members to discuss and handle an orderly transfer of the peer review results with the purpose to use these as basis for transfer and twinning activities. The meeting will also consider possible adjustments to the peer review process to maximise their integration in and usefulness for the IPF project. PAGE 11