Evaluation of Formas applications 1. Review of applications general The mission of Formas is to promote and support basic research and needs-driven research in the areas of the Environment, Agricultural Sciences and Spatial Planning. The research that is funded shall be of the highest scientific quality and relevance to the areas of responsibility of Formas from a national and international perspective. Formas may also support development projects to a limited extent. The applicant is encouraged to be thorough with the structure and clarity of the application, as the review will be based solely on the information provided in the application. Applications are assessed factually and impartially and in accordance with the instructions and guidelines provided to applicants in the call text. Applications are evaluated by an international review panel that consists of active researchers and users of research results who are qualified to assess potential societal benefit. Researchers comprise the majority. The panel possesses the expert knowledge required to review natural and social sciences as well as research within the areas of health, technology and the humanities within the areas of responsibility of Formas. The panel also has the capability to review multidisciplinary and interdisciplinary research. Each application is evaluated according to three criteria for scientific quality (research question, methods and performance, scientific competence) and two criteria for societal relevance (the potential societal benefit of the research question and communication of the research to stakeholders/users). All of the criteria must be addressed in the application and applicants are encouraged to clearly and specifically relate the application to these criteria. Each of the scientific quality and societal relevance criteria are considered to be of equal importance and the overall assessment of the application is made with no predetermined weighting of the five criteria. Applications for targeted calls are assessed according to the evaluation criteria stated in the call text, but if no additional criteria are stated the usual Formas criteria for scientific quality and societal relevance apply. With regard to the relevance criteria, the applicant is encouraged to explicitly relate the proposed research project and the questions to be addressed in a wider perspective and to describe the relevance this can have for society, both nationally and internationally, over the short and/or longterm. The responsibility is on the applicant to factually and realistically discuss these aspects in the application in a way that makes it possible for the reviewers to assess whether the proposed research and its results relate well to the societal value, as described by the applicant, in relation to the evaluation criteria. Note that in cases where the application concerns basic research where societal value is not directly linked to a specific application and/or is not regarded to be of immediate value but is instead more system oriented, applicable to different potential applications and/or is expected to be of value in the future, the applicant must also present explicit and clear arguments describing the role of the basic science, wide system oriented perspective and/or long-term effects in relation to the relevance criteria: potential societal value (with examples of potential application areas in cases where several of such may have long-term relevance) and communication with stakeholders/users (where for basic science projects these may also be applied scientists and national or international research organisations). A basic research proposal solely having potential long-term societal value and a communication plan that primarily involves other researchers can still be granted funding if the basic research described is of high scientific quality. However, an application that receives high scores for the societal relevance criteria but low scores for scientific quality cannot be granted funding. 2. Grounds for early rejection or refusal of an application The Formas Secretariat will reject, without review, applications that clearly do not fall within the responsibility areas of Formas or the framework of the call. The Secretariat will also reject applications that have not fulfilled the eligibility requirement or are incomplete, i.e. applications for
which mandatory information is lacking from the application form or appendices. Decisions about early rejection are taken by the Formas Secretary General. More information about grounds for early rejection or refusal of an application can be found in General instructions for applicants to Formas. 3. Evaluation criteria 3.1 Criteria for Scientific Quality Research topic Scientific significance of the objectives of the research. Originality and novelty of the purpose, theory and hypotheses. Possibility of significant scientific results. Objectives consistent with the call. Multidisciplinary and interdisciplinary approaches must be regarded as being advantageous when appropriate for the research topic. Gender perspectives, class, ethnicity or other social aspects must be included when appropriate to the research topic. Methods and performance Feasibility and suitability of the scientific methods. Innovativeness of the methods. Specific and realistic work plan. Specific and realistic plan for scientific publication and dissemination of information. Coordination of the project and the research group. Suitability of multidisciplinary and interdisciplinary approach. Ethical considerations Feasibility of the budget in relation to the structure of the project and the anticipated results. The feasibility and suitability of scientific methods must primary be considered. Scientific competence Scientific quality of the publications. Ability to carry out the project according to the project plan. Experience of supervision of younger researchers (postdoctoral and doctoral students, as well as degree project students for applicants who are young researchers themselves). Experience of project leadership. National and international activities, including projects, networks, commissions, honorary commissions, participation in or organisation of workshops and conferences. Interest in, experience and ability to disseminate research and research results to stakeholders/users. Strength and competitiveness of the research group. The quality of the scientific publications must be assessed taking into account the standards within each scientific field. When several researchers collaborate the scientific competence of the individual researchers and the collective scientific competence of the group is assessed. For Mobility Starting Grants the strength and competitiveness of the research environment at both the home and the host university must be assessed.
3.2 Criteria for societal relevance Potential societal value of the research topic The research topic addresses important social/sector issues within the focus areas of the call (all of the responsibility areas of Formas in the annual open call), nationally and/or internationally. The project may, over a short-term or long-term perspective, contribute to sustainable development nationally and/or internationally. Stakeholder/end user needs have been taken into account in the design of the project. The objectives are consistent with the call. Awareness of stakeholder/user needs may comprise references to e.g. directives, environmental objectives and strategies, as well as discussions with relevant stakeholders/users. Communication with stakeholders/users Description of relevant stakeholders/users. Specific and realistic plan for the involvement of relevant stakeholders/users in the project and for the communication of the research and its results to these parties. Stakeholders/users must be regarded in a broad sense as actors outside and/or sometimes also within the scientific community (depending on whether the project has a more basic research or applied research character), nationally as well as internationally, who can benefit from the research results or facilitate the future use of the results in society. Communication with stakeholders/users may take different forms and have different time scales depending on the research topic addressed, but should include different forms of dialogue with stakeholders and potential users of the research and the research results. See Support to address the communication criteria. 4. Gender equality There shall be no discrimination due to gender in the processing of applications. The scientific quality and relevance of the application takes precedence over the equality aspect, but in cases of equal assessments priority shall be given to the minority gender. The review panel should also try to assure that the gender balance for the main applicants of the projects nominated to be granted funding does not deviate in a significant and unwarranted way from the gender balance for the main applicants of all applications allocated to the review panel. When the scientific productivity of an applicant is evaluated, only years in which research has actively taken place should be taken into consideration. This means for example that a period of parental leave, illness, military service or public service must be disregarded. 5. Evaluation process Assessment of competence and conflict of interest Before the evaluation work begins, all panel members assess their own competence to review each of the applications allocated to the review panel. They also assess if any conflict of interest situation exists. Competence to review the applications is made using a three level scale: 3 = higher competence 2 = medium competence 1 = lower competence
Allocation of reviewers for each application Formas usually appoints four reviewers for each application. These must be panel members who have the highest competence to review the application in question. One of the reviewers is appointed as the rapporteur. The rapporteur is responsible for briefly summarising the application at the review panel meeting and for compiling the written opinion statement of the review panel. External reviewers External reviewers are used in cases where not enough panel members have sufficient competence to review an application. These may be panel members from another Formas review panel. The application is still reviewed by four members of the review panel, but the assessment of the external reviewer or reviewers is used as a guide. When an external review is carried out points and a written assessment are made for each of the review criteria. The Formas conflict of interest principles also apply to external reviewers (Formas General Guidelines). Points scoring scale The reviewers evaluate the applications they have been allocated, award points scores for the evaluation criteria for scientific quality and societal relevance, and write brief comments on the strengths and weaknesses of each application. These comments are mandatory and are used to facilitate the discussion at the review panel meetings and as an aid to the rapporteur in the compilation of the written statement for the application. The individually awarded points and written comments form the working material of the panel and are not made available to the applicant. The review panel members award points for each of the evaluation criteria according to the following scale: 7 Outstanding. The application successfully addresses all relevant aspects of the criterion in question. Shortcomings are insignificant. 6 Excellent. The application successfully addresses all relevant aspects of the criterion in question. Shortcomings are minor. 5 Very good. The application addresses the criterion very well, but with some notable shortcoming. 4 Good. The application addresses the criterion well, but with several notable shortcomings. 3 Acceptable. While the application broadly addresses the criterion, but there are considerable weaknesses. 2 Poor. The application addresses the criterion in an inadequate manner, or there are serious inherent weaknesses. 1 Insufficient. The application fails to address the criterion under examination or cannot be judged due to missing or incomplete information. Review panel meeting Each review panel meets to discuss and rank the applications in order of merit. The points awarded by the individual panel members and the written comments form the basis of the panel discussion, which in turn forms the basis of the collective panel scoring and statement. Only applicable to the Formas annual open call The applications in the annual open call are ranked at the panel group meeting according to the different types of grants available: Research and Development Projects Research and Development Projects for Future Research Leaders Mobility Starting Grants for Young Scientists
Collective review panel statement Each project leader receives a written report containing the collective assessment of the application by the review panel. The collective statement reports the assessment in terms of: The points score for each criteria. The overall points score. Written comments summary. Budget allocation within the annual open call The Formas budget for Research and Development Projects is allocated among the eight review panels based on the amount of applications received, which means that the review panels can make a final ranking in order of merit of these applications at the review panel meeting. For the two funding forms Research and Development Projects for Future Research Leaders and Mobility Starting Grants the Formas budget is a joint collective budget for all review panel groups. Each review panel ranks the applications in order of merit for the respective grant form. Afterwards there is a joint overall ranking of the applications for both of these grant forms, based on the rank order that was agreed at each of the respective review panel meetings. For applications nominated to receive grant funding for Research and Development Projects and Research and Development Projects for Future Research Leaders the review panels make an assessment of whether the budget that has been applied for is reasonable in relation to the activities described in the research plan and the anticipated results from the project. This means that the review panel has the mandate to decide whether a reduction to the budget applied for shall be implemented for both of these grant forms. In the Formas annual open call the budget determines what proportion of the submitted applications can be awarded funding. In the review process the collective points score for a number of applications is not sufficiently high to be able to be granted funding in the context of the quality of the competition. This is not a matter of an individual detail, but is instead based on the collective assessment. These applications receive a standard text as the summary written comments in the review panel collective statement 6. Review panels Review panel members Members of the review panels for the Formas annual open call are appointed each year. For targeted calls a review panel is appointed that has the competence to encompass the subject area in question. Review panels are made public on the Formas website after the Formas Scientific Council has reached its decisions for the call. The chair and vice-chair, supported by Formas, are responsible for ensuring that the work of the review panel is carried out in accordance with the general mandate of Formas, the regulations governing conflicts of interest, the guidelines for review of applications and otherwise in accordance with good ethical practice (Formas Ethical Policy). The vice-chair acts as chair if the chair is not able to lead the work. Terms of appointment It is possible to be a Formas annual open call review panel member for a continuous period of four years. The appointment of panel members is however revised each year. The chair may serve for an additional one year term. When five years have elapsed after a previously concluded term, a person may again be appointed as a member of a conventional Formas review panel. It is the intention of
Formas that the members of the review panels should be replaced successively, so that only a minority of panel members are replaced each year. 6.1 Guidelines for the composition of Formas review panels Review panels comprise a chairperson, a deputy chairperson and a suitable number of panel members for the call. By suitable is meant the number of panel members necessary to satisfy the breadth of competence required for reviewing the applications submitted in response to the call. The panel members have the mandate to review both the scientific quality and the relevance within the responsibility areas of Formas. The chairperson is a researcher who has professorial level or equivalent competence, has a broad knowledge of the subject area, has experience of strategic work and has insight into the societal value of the research from a sustainable development perspective. The appointed deputy chairperson is a representative of the users of the research results. The deputy chairperson must have a broad knowledge of the user value and societal value of the research from a sustainable development perspective, as well as having significant experience of strategic work. The majority of panel members are active researchers, within or outside of Sweden, who have been selected to become members of the review panels primarily because of their competence to review the scientific quality of the applications. The researchers must have a broad knowledge of the subject area, at or equivalent to senior lecturer/associate professor level, and must be able to work strategically. A minority of panel members are users who have been selected to become members of the review panels primarily because of their competence to review the relevance with the Formas areas of responsibility. They may be active within or outside of Sweden. The users should have documented experience of research, e.g. by possessing a doctoral degree, previous review experience or strategic development experience. The users must have an overview of the short-term and long-term needs of society and have experience of strategic work. The review panel as a whole must be able to evaluate applications within the entire subject area spanned by the applications allocated to the panel, i.e. the panel must have a subjectwide coverage and be able to evaluate basic research and needs-driven research, as well as user and societal benefit. The final decision regarding the composition of the review panels is not made until the applications have been received. If necessary external expert reviewers may be recruited to supplement the competence of the review panels. Each review panel must have an equal gender distribution with each gender having at least 40 per cent representation. The chairperson and deputy chairperson in the Formas review panels must together constitute an equal gender distribution. 7. Managing conflict of interest All of the reviewers involved in evaluating applications on behalf of Formas must read the Formas Ethical Policy (Formas General Guidelines) before embarking on their assigned task. Applications made by review panel members may not be evaluated by that review panel, regardless of whether the applicant has the role of project leader or another participatory role. If a panel member wishes to submit a grant application to Formas and no other relevant review panel exists, that review panel member will be replaced. Review panel members must notify Formas as soon as possible of the possibility that they may intend to submit an application.
A fundamental requirement of the work carried out by the review panels is impartiality. Provisions governing conflict of interest can be found in sections 11 and 12 of The Administrative Procedures Act (1986:23). Conflict of interest is regarded to exist in the following instances: The matter concerns the panel member or a person close to the panel member, or the outcome of the matter can be expected to result in significant advantage or disadvantage for the panel member or a person close to the panel member. The panel member or a person close to the panel member is a representative of, or works at, the same department or company as the applicant, or is a representative of another party for whom the outcome of the matter may result in significant advantage or disadvantage. The panel member has an ongoing or recently completed close collaboration with the applicant. There is also conflict of interest if any other particular circumstance exists that may influence the credibility that a panel member is impartial in the matter. Examples of such circumstances are friendship, rivalry or financial dependence. Members of the review panels are obligated to take into account and notify conflict of interest and on their own initiative report if there may be any circumstances that may be regarded to influence their opinions. If conflict of interest exists the panel member must not participate in processing and reviewing the application in question and must leave the meeting room during discussions concerning the application. Meeting minutes detailing conflicts of interest must be kept during panel meetings.