Alpbach Technology Forum, The Efficiency of RTI Investments, 26 August 2011 EU RESEARCH : VALUE FOR MONEY?

Size: px

Start display at page:

Download "Alpbach Technology Forum, The Efficiency of RTI Investments, 26 August 2011 EU RESEARCH : VALUE FOR MONEY?"

Dortha Della Short
6 years ago
Views:

1 Alpbach Technology Forum, The Efficiency of RTI Investments, 26 August 2011 EU RESEARCH : VALUE FOR MONEY? Wolfgang Burtscher DG Research and Innovation European Commission

2 Structure PART I. About the Framework Programme PART II. Evaluating EU R&I funding PART III. Illustration of evaluation evidence PART IV. Future strengthening of evaluation system

3 Part I About the Framework Programme

4 EU research: Characteristics of the Framework Programme (1) Most important multinational research programme world wide Project funding only True common put no national quota Competitive process with annual calls and peer review proposal evaluation Main focus on collaborative projects with partners from several countries Open to academia and industry alike Open to the world, not restricted to Europe

5 FP7 Structure Cooperation Collaborative research Multi-partner projects, top-down agenda Ideas Frontier Research Individual projects, bottom-up agenda Capacities Research Capacity Sharing of infrastructures People Marie Curie Actions Mobility programmes for post-docs, Training courses,

6 FP7 Budget ( million)

7 FP7 Facts and Figures (2007 to 2010) proposals received, applicants proposals retained, participants Out of these: Universities 30% Industry 25% Research Organisations 23% 16,6% of all participations are SMEs

8 Part II Evaluating EU R&I funding

9 Evaluating R&I investments is difficult Time lags some results have an impact in the longer term Multiple kinds of impacts R&I have an impact in several dimensions, which might not be easily identified: Scientific impacts Technological impacts and impacts on innovation Economic impacts Social impacts Environmental impacts Structural impacts on ERA Impacts on EU policies European Added Value Impacts on SMEs Attribution R&I is often funded from different sources Valuation it is difficult to provide a monetary value of impacts to make them comparable

10 New context for evaluating R&I in Europe - Economic crisis is threatening expenditures on research & innovation, which need to demonstrate high impacts and economic returns - Next Multiannual Financial Framework will be focusing on instruments with proven European Added Value and value for money

Assessment FP7 Interim Evaluation Thematic

11 Over 20 years of efforts in evaluating Framework Programme FP 5 FP 6 FP 7 Annual monitoring 5 year assessment FP6 ex post FP7 Ex ante Impact Assessment CSF Ex ante Impact Assessment FP7 Interim Evaluation Thematic level evaluations National Impact studies

12 Framework Programme Evaluation System Consisting on ex-post evaluation and ex-ante impact assessment Increasingly geared to capture value for money Traditional evaluation methods: Expert panels, Interviews, Surveys of programme participants New methodologies: Linking of different data sources to programme participation (bibliometrics, networks analysis, innovation impacts) Ex-ante modelling of impacts

13 FP7 Interim Evaluation Carried out in 2010 Group of 10 external experts Covered FP as a whole Final report online Political follow-up Commission Communication Council Conclusions Report by the European Parliament

14 FP7 Interim Evaluation: Key Message FP7 is on course and is clearly making a significant contribution to European science and the development of the European Research Area There are acknowledged difficulties in some aspects of its implementation, but it is important to applaud what is good about it

15 Evidence used for the Impact assessment of Horizon 2020 Robust quantitative and qualitative evidence : ex-post and interim evaluations (FP6, FP7, CIP, Marie Curie, ERC, ) foresight and forward looking studies statistical data (FP, CIP, Community Innovation Survey, ) analyses of science, technology and innovation indicators; (EC, ESTAT, OECD..) econometric modelling exercises; (NEMESIS, DEMETER ) reviews of academic literature on market and systemic failures and the impact of research and innovation, and of public funding for research and innovation; sectoral competitiveness studies; expert panels and expert hearings; etc. On-line surveys among FP and CIP beneficiaries.

16 Part III Illustration of evaluation evidence

17 Outputs and reach Framework Programme research funding supported thousands of projects and participants: - involving different types of top (A-team) research actors : - business enterprises, - research centres, - higher education institutions - NGOs - from a large number of countries and regions: - EU 27 Member States and their regions, - associated and candidate countries third countries including BRICs

18 FP Evaluation Studies: Top 50 Participants Top 50 Participants account for 25% of FP7 funding Strong indication that FP7 is not a closed shop Rank Organisation Name Countr y Participations EU Contribution in Mio 1 CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE FR ,0 2 FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V DE ,1 3 COMMISSARIAT A L' ENERGIE ATOMIQUE FR ,7 4 MAX PLANCK GESELLSCHAFT ZUR FOERDERUNG DER WISSENSCHAFTEN E.V. DE ,4 5 THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF CAMBRIDGE UK ,8 6 ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE CH ,4 7 THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF OXFORD UK ,7 8 FONDATION EUROPEENNE DE LA SCIENCE SUP 9 93,5 9 EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH CH ,7 10 IMPERIAL COLLEGE OF SCIENCE, TECHNOLOGY AND MEDICINE UK ,9

data based on 20 studies of additionality of EU support; national programme data

19 Projects Additionality % of FP6 participants who did/would abandon the project without FP funding in comparison with national programmes participants Source: FP data based on 20 studies of additionality of EU support; national programme data based on studies for Member State programmes in Austria, Belgium (x2), Finland and Norway.

20 Leveraging private funding The multiplier effect of the FP7 Risk-Sharing Finance Facility, is expected to be over 30 between the EU contribution and the additional leveraged investment in RDI.

21 Impact on R&I capabilities Different FP ex-post impact studies demonstrate that EU funding, i.e.: increased FP participants ability to network, administrating and managing international projects (Study on FP6 behavioural additionality) increased skills and research capabilities of participants key research staff (Study of the impact of FP6 on new Member States) A UK evaluation of FP6 and FP7 found that the FP has a big impact on the nature and extent of UK researchers' international relationships and networks, as well as on their knowledge base and scientific capabilities.

22 Structuring effects: durable changes in the european RTDI landscape Scientific excellence is promoted via EU-wide, competitive ERC grants Creating research infrastructures of pan-european importance (ESFRI roadmap) Developing joint industrial strategic research agendas (ETPs and JTIs) Bringing together fragmented national research funding to tackle important societal challenges (ERA-NETs) Creating persistant R&I networks through the FP collaborative research Increasing the attractiveness of Europe as a place to carry out research (ERC, MCA) Influencing the design of Member State research policies, especially in the new Member States

23 Scientific impact - FP projects produce thousands of scientific publications - FP6 lead scientists have better scientific impact than scientists without FP participation (Study on FP6 network effects) - According to a German evaluation of FP6 scientific personnel participating in FP6 stated that a substantial part of their publications was due to their participation in the FP.

24 Impact on technology, innovation and competitiveness Firms motivation for participation: knowledge + networking opportunities, less direct commercialisation Nevertheless, FP produces large numbers of patents and innovations Framework Programme participants are more likely to apply for a patent :

25 Impact on growth & jobs On the basis of the NEMESIS econometric model, the long-term FP7 macro-economic impact was estimated at an extra 0.96 percent of GDP, an extra 1.57 percent of exports, and a reduction by 0.88 percent of imports The long-term employment impact of FP7 was estimated at 900,000 jobs, of which 300,000 in the field of research.

26 Policy impact Past Framework Programmes have an impact on the decision-making process, i.e.: At the international level, EU research related to climate change contributed to the International Panel on Climate Change Water and soil projects played a large role in the formulation and implementation of the Water Framework Directive. According to an Irish evaluation of FP6 each project counted, on average, 0.4 new or significantly improved regulation or policy.

27 Part IV Future strengthening of evaluation system

28 Monitoring & Evaluation system in Horizon 2020 Learning from the past, the new Framework Programme will have better monitoring and evaluation tools : - Comprehensive : evaluation coverage of all CSF action lines, with a detailed timetable for evaluation. updated and revised annually, taking into account new developments - Well-timed : A comprehensive interim evaluation in 2017 (3 years into the programme), and a full-scale ex-post Evaluation in 2023 (2 years after the end of the programme) - Harmonised : common templates, methodologies and indicators - Strong focus on throughput, output and impact - Supported by an appropriate data archive, experts, dedicated research activity, and increased cooperation with Member States and Associated States - Valorised through appropriate dissemination and reporting: transparency of the evaluation process is a key element of an overall strategy for full accountability.

Towards a Common Strategic Framework for EU Research and Innovation Funding

Towards a Common Strategic Framework for EU Research and Innovation Funding Replies from the European Physical Society to the consultation on the European Commission Green Paper 18 May 2011 Replies from