EU HEALTH PROGRAMME Call for Proposals 2008 Survey of participants' satisfaction
Call for Proposals 2008 Survey of participants' satisfaction Executive Agency for Health and Consumers 2009
Introduction The Executive Agency for Health and Consumers (EAHC, formerly the Public Health Executive Agency) was created on 1 January 2005. It is based in Luxembourg. In 2008, the Agency's mandate was prolonged and expanded to include actions in consumer protection and training for safer food. The EAHC implements the EU Health Programme, the Consumer Programme and the Better Training for Safer Food initiative. The Agency provides a professional service in performing the tasks and activities entrusted to it by the European Commission and it works closely with the Health and Consumers Directorate General. The EAHC manages relations with some 2200 beneficiaries involved in over 200 projects in the field of health; it has about 50 staff members with an administrative annual budget of 4.5 million euro (year 2008). The Second Programme of Community Action in the Field of Health 2008-2013 came into force on 1 January 2008 and is intended to complement, support and add value to the policies of the Member States. It contributes to increased solidarity and prosperity in the European Union by protecting and promoting human health and safety and by improving public health. In the new Programme, the range of financing mechanisms was enlarged to include: cofinancing of projects intended to achieve a Programme objective, cofinancing of the operating costs of a non-governmental organisation or a specialised network, cofinancing of conferences and joint financing of common activities carried out by the Community and one or more Member States or by the Community and the competent authorities of other countries participating in the Programme together. In October 2008 EAHC carried out a satisfaction survey among the applicants to the 2008 Calls for Proposal under the Second EU Health Programme. This survey was administered via the EAHC internet site and invited all those who had submitted a proposal regardless of whether it had been selected for co-funding or not to participate. The main objective of the survey was to assess the user-friendliness of the callrelated documents, namely the application forms and guide for applicants and to identify the main problems applicants found when filling in the forms. Over the past years EAHC has developed a range of support for potential applicants. This includes particularly a network of National Focal Points (NFP), which is made up 3
of members of national administrations (e.g. Ministry of Health) who have been specifically trained by EAHC to serve as the first contact point at national level concerning the Calls for Proposals and the related procedures. In addition, when a Call is launched, EAHC offers a help-desk via telephone and e-mail. Thus this satisfaction survey was also designed to evaluate the support given by the NFP and the EAHC help desk. Lastly, the survey was also used to gather different suggestions or ideas related to the submission of applications. Much insight was gained from the answers received. The EAHC plans to repeat such a survey every year after the Calls for Proposals are closed. Methods The method used was an invitation to all applicants to the Call 2008 (255 applicants) to fill out the survey online on the EAHC Website, through a link, with restricted access, sent by email. The questionnaire, in the annex, consisted of 13 questions, organised as follows: 3 questions for each type of application form, 4 questions related to the guide for applicants, 3 questions about the NFP support and 3 about the EAHC helpdesk. The link to the questionnaire was active for 10 days, from 1 to 10 October and had a response rate of 30% (76 replies/ 255). During the period of the survey, only applicants with proposals recommended for funding were informed of the decision. The other applicants with proposals on the reserve list or not selected for funding did not know the results of the call. Results From the respondents to the survey, the majority of applicants had submitted a proposal for a project (68%). The rest applied for grants for conferences (16%), operating grants (13%), and only 3% of the answers corresponded to applicants for a joint action. Application form Concerning the question on how easy it was to complete the application forms, half 4
of the applicants considered it easy or very easy to fill-in and the other half found it difficult or very difficult. In relation to the specific problems experienced by the applicants, the results are presented below by type of instruments, projects or joint actions and, once the application form had the same structure, conferences and operating grants. Projects or Joint Actions As shown in the figure below the financial management data section appears at the top of the problems encountered by the applicants with 18%, followed by difficulties in the description of the work packages and Policy and context relevance, with 12%. Figure 1 Specific problems experienced when filling in the form for project application If we group the problems experienced by the applicants following three different sections of the application form, according to the type of problem, we can show that: 55% of applicants experienced problems in technical aspects (description of the work packages, policy and context relevance, management of the project, technical aspects of the project, specification of the project, and executive summary), 26% experienced problems in the financial aspects of the application form (Financial viability information and financial management data) and 19% found problems in administrative aspects such as the participation in EU funded projects, overview of the project, partner information or supporting documents. 5
In the table below are mentioned the most common specific type of problems listed by the applicants and at same time the proposed solutions (the comments are taken directly from the questionnaire, they were not edited). Problems experienced Hard to guess what information exactly expected under different headings of the specification, management, policy and context relevance, etc. Some of the requested information would fit to several of the boxes No flow in the form - Difficult to describe in a logical series of steps what the project will do and how it will make a difference to the population health. Hard to fill in the form when constantly having to refer to the guidance. Finance system part of the form worked well and was easy to use. Definitions of outcomes and output/deliverables offered in the "guide of applicants" are more confusing than explanatory There is too little space to enter information in the timetable Practical problem with the Declaration of honour for the associated partners, due to the fact they are incorporated in the application form. Provision of space seems to be insufficient to describe the project issues clearly and not compatible with the number of characters provided Difficulty to identify the sections which were deemed incomplete after attempting to valid the form Problems when: adding partners; picking/unpicking the boxes To find partners, with specific expertises to be involved in the partnership Proposed solution Guidelines will be revised The application form was revised in order to be more user-friendly. To be revised The timetable will be automatically filled in For 2009, the Declaration of honour will be automatically filled with the data for each partner entered in the form. Once the data is entered, it will be enough to print the forms and sign them. (ongoing testing) All forms will use a font that takes the same space for all letters or numbers (E.g. Courier new) A new script will be added in order to solve this problem More tests will be performed on the different platforms A tool for partnership will be available. (testing ongoing) 6
Conferences When applying for conference funding, 29% of the applicants found specific problems in filling in the financial overview, 21% of applicants with problems in entering information concerns the management of the conference. 14% of the applicants also experienced problems dealing with the supporting documents, human resources information and 7% of applicants had problems related to additional useful information, specification of the conference and general information topics.. Operating Grants In figure 2 below we can observe the results obtained concerning the problems applicants found when applying for an operating grant. Figure 2 - Specific problems experienced when filling in the form for an operating grant application 7
Guide for applicants The majority, 91%, of applicants considered the guide for applicants as useful or very useful, (see figure below) but a few others considered it too big and heavy, proposing to split the guides per financial instrument. The advice on developing the proposal, copy/paste approach and the time to fill in the form was also considered misleading. The applicants proposed the inclusion of some good examples on how to describe the different topics (objectives, outcome models, etc). Figure 3 Usefulness of the Guide for Applicants Support from the National Focal Point and EAHC Helpdesk 96% of the participants in the survey confirmed that they consulted their NFP when preparing the proposal and 92% consulted the EAHC helpdesk. The rates of satisfaction are very high both for the support given by the NFP and by the EAHC helpdesk. In the case of the support of the NFPs, 85% of the participants found the support good or very good. For the support received by the EAHC helpdesk, 92% of participants classified the support as good or very good. The most frequent type of support required was concerning administrative matters, both for the NFPs and EAHC helpdesk, followed by support on project design in the case of NFP and 8
financial issues for the EAHC helpdesk, as shown in the figure below. Figure 4 Support received by NFP network and EAHC helpdesk Other comments received concerning the application to the call 2008 were: The application should be more user friendly and precise in its requirements The language should be simpler In the application for an operating grant, the structure of the form and the requested information/description of actions were too project oriented Make it slimmer for the first application. Go for details during the second step The results of the evaluation are delayed too much. Conclusion and future actions We can conclude that the survey was a positive experience that could be repeated after each Call. The response rate was sufficient to consider the sample representative of the applicants under the Call 2008, even though the number is fairly low and the time frame for the data collection was short. The assessment of the application forms is an important feedback in terms of technical, financial and IT issues, which helps in the preparation of the application forms for the call 2009. For next year, if possible, the satisfaction survey will be linked to the application form, as a pop- up application which generates the questionnaire. This tool will allow real time and independent feedback by each applicant. 9