ESF-logo Peer Review at ESF Farzam RANJBARAN Head of Unit, Corporate Science Operations European Science Foundation Thirteenth Symposium European Network on Research Careers 13 October 2009, KoWi, Brussels
Outline About ESF Peer Review at ESF Food for thought/impulse for discussion 2
About ESF Mission The European Science Foundation provides a common platform for its Member Organisations in order to: advance European research explore new directions for research at the European level Through its activities, the ESF serves the needs of the European research community in a global context. 3
About ESF Independent, non-governmental organisation Established in 1974 All fields of science 80 MOs in 30 countries Offices in Strasbourg, Brussels, Ostend ESF budget: 53M in 2008 This leverages several Bn of research and networking Staff: 104 in 2008 ESF headquarters, Strasbourg Marine Board, Ostend Research Conferences Unit, 15th floor, Tour Louise, Brussels 4
Strategic Plan 2006-2010 SCIENCE STRATEGY Forward Looks Science Policy Briefings Exploratory Workshops SCIENCE SYNERGY EUROCORES Research Networking Programmes Research Conferences SCIENCE MANAGEMENT Peer Review support COST contract Coordination of other external Contracts; ERA-NETS MO Fora A dedicated Forum on Peer Review 5
MO Fora Output-oriented discussion platforms for Member Organisations (MO) to develop joint actions on specific issues, involving others as appropriate Actions: to benefit MO strategy development and/or lead to the development of Best practice e.g. peer review Common procedures Cooperative activities Time-limited activities (two years) Target groups: ESF Member Organisations /activities/mo-fora 6
Existing Fora Research Integrity Peer Review Research Careers Evaluation of Funding Schemes and Research Programmes Completed: Promoting Internationalisation of Social Sciences in Central and Eastern Europe 7
ESF Peer Review MOST CRUCIAL ELEMENT OF OUR OPERATION It determines, to a very large extend, the quality of our programmes and services It binds several of our instruments together It sets increasingly high expectations from clients and stakeholders Peer Review at the ESF has evolved over the last two decades into a core competency and is now becoming a specialized support to our MOs Numerous specific requests from our MOs and others in addition to the needs of our instruments 8
External Peer Review support Motivation and needs: Broadening of National practices internationalisation of selection criteria and evaluation protocols Broadening of Expert Base Benchmarking of national schemes Ensuring independence when several players are involved: objectivity and fairness Outsourcing Peer Review and Evaluations 9
Examples: Helmholtz Association, Germany review of research fields: Energy, Key technologies and Structure of Matter Royal Netherlands Academy of Arts and Sciences (KNAW); reviewers for the Academy fellowships Danish National Research Foundation research proposals UNIK and Centres of Excellence. And, Mid term review of Centres of Excellence Cyprus National Research Promotion Foundation ANEP: Spanish National Evaluation and Foresight Agency (Agencia Nacional de Evaluación y Prospectiva) for Universities and Research 10
Examples: ERA-Nets: European PolarCLIMATE programme under European Partnership in Polar Climate Science European Space Agency announcement of opportunities for three programmes: about 300 proposals IRCSET INSPIRE and EMPOWER Schemes: a review of Post Doctoral fellowship applications Bulgarian Academy of Sciences Expert evaluation of 69 research units in 8 Divisions 45 scientists involved in 4+1 panels Expected to submit the final report by end 2009 11
MO Forum on Peer Review Aim of this Forum: sharing of experiences and developing common good or better practicies in PR Participation organisations: 1.FWF, Austria 2.FWO, Belgium, 3.GACR, Czech Republic 4.Danish RC for Technology and Production Sciences 5.Academy of Finland, 6.DFG Germany 7.Health Research Board, Ireland 8.INFN, Italy, 9.NWO and KNAW, the Netherlands 10.Foundation for Science and Technology, Portugal 11.Swedish RC, 12.Swiss National Science Foundation 13.Slovakia, 14.EPSRC, UK Observers: EC, ERC, NSF, Telethon 12
MO Forum on Peer Review ACTION PLAN: ACTION 1: ESF Peer Review Guide - A Management Tool Box for Research Funding Organisations ACTION 2: A pilot study on practices regarding incentives for peer review ACTION 3: Referee Databases: quality of contents ACTION 4: Web Bank of Peer Review Practices on the ESF web site 13
MO Forum on Peer Review Current activity Just received a 2-year extension Preparation of a comprehensive survey on peer review practices across our MOs: General organisation of peer review processes Types of research classification systems Methods of quality assurance Handling of interdisciplinary and high risk proposals Use of incentives Etc. To be launched at the beginning 2010 14
Food for thought Personal points of view: Interdisciplinarity: How to adequately review MIT proposals? Either don t promote them, or be prepared to adequately define and evaluate them Interdisciplinary character: Host versus Guest Disciplines, or Driving versus Assisting Disciplines Most crucial: appropriate ownership If not really necessary, avoid engaging multiple planels, or dedicated Multidisciplinary panels. 15
Food for thought Personal points of view: High-Risk proposals: Aime for Breakthroughs or frontier research -> transformative science Aim for smaller and faster Allow discretions to programme managers and officers (NSF) Define Grand Challenges Provide guiding framework and disseminate Help identify disciplinary barriers 16
Food for thought Personal points of view: Is there a real problem with Peer Review? PR is not Self-Organizing enough, why? - Is there too much science? - Are there too many scientists? - Perhaps there is not enough money! - Or time spent on PR is considered time lost for real science!.. 17
Food for thought Personal points of view (perhaps too simplistic): Non-reconcilable forces may be at work: 1- Demands are growing 2- Availability and endogenous interests are not growing Hence, automation and exogenous incentives are called for This may be driving PR from being an intellectual and scientific dialogue to a mechanical recipe-type and almost commercial endeavor!! 18
Food for thought Personal points of view: Then, how can PR be made more selforganizing? By re-capturing and repositioning its true value! As long as funds are insufficient to give to all: selecting good science for funding would require better science to do the judging Publishing and grant making need PR but they are valued very differently from it Tremendous role for early career scientists Need for real attention to gender equality 19
ESF-logo Excellence Openness Responsiveness Pan-European Ethical awareness Human values