Terms of Reference. For the external evaluation of the following Action contre la Faim s. Programme funded by: ECHO

Transcription

1 Terms of Reference For the external evaluation of the following Action contre la Faim s Programme funded by: ECHO Intervention d urgence en Sécurité Alimentaire et Nutrition suite au tremblement de terre dans la région de Port au Prince (Haïti) le 12 janvier 2010 References of the contracts: ECHO/-CR/BUD/2010/01018 TORs produced on the 19/06/2010 ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

2 I. CONTRACTUAL DETAILS OF THE EVALUATION I.1 Evaluation Dates Expected Start Date: * End Date: 09 / 08 / / 08 / 2010 Submission of Draft Report: 09 / 09 / 2010 Submission of Final Report: 17 / 09 / 2010 NB. ACF-UK requires reception of the ToR two months before the start date for each evaluation. I.2 Language of the Evaluation * Language of the report: French (Reports submitted in any working language of ACF-IN - English, French, Spanish - will be accepted if the consultant also covers the cost of translation to the required language). II. WORK PLAN AND TIMETABLE * Activities Working Days Briefing Headquarters in ACF-Paris 1 Travel to the mission 1 Briefing Mission, review of documents, and preparation of field work 2 Field Work 10 Meeting with others actors in the Field (Oxfam, Red Cross, CNSA, 2 PAM, ) Data Analysis and preparation of the draft report 5 Debriefing in country on the basis of the draft report 1 Travel to HQ 1 Debriefing HQ 1 Draft report submission 3 HQ feedback (5 working days) Finalisation of the report on the basis of Field, HQ, and ACF-UK 1 comments HQ approval of final report (2 working days) = Invoice payment TOTAL 28 III. BUDGET (GBPS) Budget for the Evaluation: 12,981 GBP Deadline for invoicing e.g. contract end: 21/09/10 Per Diem: Per diem is included in the consultant daily rate and therefore will not be paid sparately. Breakdown of Budget: Daily rates for consultancy are between 150 to 400 per day dependent on experience and evaluation of the ToR. Management fee is a minimum of 700 GBP but may be more depending on the scope of the evaluation and donor funding available. There is a discretionary charge for advertising on ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

3 Evaluation Budget Daily rate ( GBP) units total Consultant name Mr Rene Grojean ,080 International Travel & Visa 1, ,640 Local costs (transport and communication) Management Budget Management fee 700 Upgraded advertising charityjob.co.uk add Total Budget TOTAL 12,981 GBP Exclusions: Accommodation, food and local transport will be provided by the mission at field level. Insurance and any materials to carry out the evaluation e.g. laptop, must be provided by the consultant (no exceptions will apply). In order to comply with donors requests, a quotation (or draft contract) must always be provided to be annexed to the final reporting information. ACF-UK will flag HQ in case any aspect of the budget cannot be respected. IV. C ONTRACTUAL DETAILS OF THE PROGRAMME Please see Annex 1 for an overview of the current context and ACF presence in the region / country. Overview * ACF has been working in Haiti since 1985 and has developed a long-term project strategy in the Water, Sanitation, Hygiene (WASH), Food Security and Nutrition sectors based on capacity building of local authorities. Furthermore, ACF has also launched disaster risk reduction activities. ACF has a permanent presence in Port-au-Prince, Gonaives and Portde-Paix to implement development type projects and has maintained an emergency capacity through the pre-positioning of contingency stocks in both locations and a human resources network of trained national staff readily available through the country. IV.1 General Objective * Respond in emergency to immediate needs and prevent the mortality of the population affected by the earthquake. IV.2 Specific Objectives and Activities * Treat and prevent the degradation of the acute malnutrition in the regrouping sites through food aid and a nutritional problem ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

4 Results: Result 1: Blanket Supplementary Feeding is assured for children from 6 to 59 months and pregnant and lactating women, for a duration of 4 months, for families affected by the earthquake. Indicators: - Number of people (children from 6 to 59 months, lactating and pregnant women) that received a nutritional supplement during 4 months - Nombre de kits cuisine individuels distribués - Nombre de personnes ayant bénéficiées de séance de sensibilisation à l hygiène et à la préparation du CSB - Total amount of nutritional supplement distributed Result 2: The quality of the nutritional intake of 15,000 families is significantly improved and the commercial activity of fresh food merchants is enhanced in the area of intervention. Indicators: - Number of families that have received monthly vouchers to buy fresh food (fruits, vegetables, meat, fish) - Number of registered sellers that furnish fresh food - Total amount of fresh food bought through the vouchers system Result 3: The nutritional situation of affected areas is assessed and specific recommendations are formulated. - 8 rapid nutritional assessment reports are written - At least 2 recommendations per report are made and reported to the concerned clusters Result 4: 4 emergency temporary centres of Programme de Traitement Ambulatoire (PTA) of severe acute malnutrition are functioning - Number of functional PTA centres close to «baby tents» - Number of SAM children with complications referred in the stabilisation centres - 16 persons are recruited and trained to take in charge severe acute malnutrition - 10 training sessions are organised within the baby tents, the partners organizations health centres on the area of intervention for the detection and registration of malnutrition cases - Indicators of discharge for severe malnutrition treatment responding to the criteria defined in the national protocol : 1. Recovery rate > 75% 2. Death rate < 10% 3. Abandon rate < 15% Activities: Result 1 : - Distribution sites identification - Registration of households beneficiaries of the food aid programme - Identification of locations for setting up community kitchens - Distribution of materials and inputs for communal kitchens and technical support for their implementation - Organization of the distributions of rations prepared for 3-5 years and supplementary plumpy for 6-36 Months - Post distribution monitoring Result 2 : - Distribution sites identification - Registration of families receiving fresh food vouchers ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

5 - Market analysis and identification of sellers participating in the program - Design of sellers payment modalities (tontine system, credit, etc) - Definition and signing of Memorandum of Understanding with participating sellers - Awareness to the vouchers utilization and to best practices of food diversification - Distribution of vouchers - Sellers payment - Market follow-up (product availability and price monitoring) - Post-distribution monitoring Result 3: - Rapid Assessment of the nutritional situation of affected populations - In the case of rapid assessment with screening of a new area, the methodology validated by the MSPP can be used using the MUAC measurement and verification of oedema - Needs assessment (whether or not to open baby tents for children under the age of 24 months or program nutritional care) - Collection of contextual information in food security, water and sanitation, health and psychosocial conditions of families - Summary report with appropriate recommendations Result 4: - Identification of four sites for the establishment of the treatment of severe acute malnutrition in the city of Port au Prince nearby baby tents taking charge of children under 1 year old and mothers - Referencing cases of severe acute malnutrition with complications to the Nutritional Care Unit Partners - Recruitment and training of personnel in the ambulatory care of severe acute malnutrition based on the national protocol for the management of acute malnutrition - Coordination of activities with the health stakeholders and / or other source of potential detection and referral of children malnourished. - Setting up training session on MUAC measurement and oedema detection to the staff working in the structures nearby PTA centres - Detection and referral of cases of severe acute malnutrition of children identified in the "baby tent" - Establishment and ongoing monitoring of activities by a nutritional program manager. V. AIM OF THE EVALUATION ACF-IN promotes and uses Evaluation as a tool to enhance operational performance as well as a way to increase ACF-IN accountability towards beneficiaries, partners and donors. The evaluation conducted must always comply with donor requirements and / or ACF evaluation policies. V.1 Objective of the Evaluation * The objective of the evaluation is to draw lessons-learned and recommendations regarding the relevance of the double-sided approach used in the urban context that is Port-au- Prince, after the January 12 th earthquake. This approach consists of: - First, blanket supplementary feeding (BSF) for families with children from 6 to 59 months and/or lactating and pregnant women - Secondly, Fresh Food Vouchers distributions to the general population where eligible BSF families are located. ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

6 The evaluation will contribute to the technical and methodological capitalisation for this type of intervention. The evaluation should as well look at program definition and implementation and propose specific and concrete recommendations, both as a response to the emergency and to the longer term needs facing the affected population. V.2 General terms and factors to consider The evaluation should consider the objectives, results and indicators outlined in the logical framework of the project. It will be undertaken as an independent examination of the background, assessment, activities and means deployed by ACF, in the implementation of the project, in order to facilitate a learning workshop and evaluation report outlining the main findings and recommendations for future interventions and decision making. The evaluation should examine the standard and quality of goods and services generated by the project, in the opinion of the beneficiaries, ACF management and technical departments, technical governmental departments and other key stakeholders. The evaluation is focused on the following results (as described above in the logical framework) and their related activities: - Result 1: Blanket Supplementary Feeding is assured for children from 6 to 59 months and pregnant and lactating women, for a duration of 4 months, for 10,000 families affected by the earthquake. - Result 2: The quality of the nutritional intake of 15,000 families is significantly improved and the commercial activity of fresh food merchants is enhanced in the area of intervention. VI. E VALUATION CRITERIA ACF-IN subscribes to the Development Assistance Committee (DAC) criteria for evaluation: Impact, Sustainability, Coherence, Coverage, Relevance / Appropriateness, Effectiveness and Efficiency. ACF-IN also promotes systematic analysis of the monitoring system and cross cutting issues (gender, HIV/AIDS etc). * Please see the attached ACF-IN Evaluation Policy & Guidelines for further details in respect of these criteria. Please see Annex 2 for details of the evaluation criteria, including questions for data collection. Please amend the criteria accordingly if the evaluation is to look more deeply into some of the criteria than others. In the case of this ACF s intervention, during the emergency phase in an urban context as complex as the city of Port au Prince, it is expected to look in depth at the following aspects: - Targeting: ACF targeted households with malnourished or at risk of malnutrition children in the earthquake affected areas. It was decided to prioritise non formal camps in the areas close to ACF office. Lists of beneficiaries were provided by local committees in these areas. Given the urban context, it would be valuable to know how well selection criteria were understood by committees; was this approach through committees appropriate?; Were criteria for location targeting clear (security conditions)? - The distribution process: a complex process was designed by ACF in this project, with four main phases: cards distribution to BSF and FFV beneficiaries; BSF distribution; FFV ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

7 redemption by beneficiaries at designed fruit and vegetable vendors in the local market and vendor payment by ACF partner. To complement the BSF and FFV component, health education was provided to the beneficiaries. Were these four phases coordinated? Were the education sessions appropriate? - Project Implementation: This project was implemented with a large number of ACF contracted staff and with some support from community committees. Due to assessment difficulties at the beginning of the project, many people were involved in the distribution process. Were beneficiary registration and distribution methodologies adequate? Were these methodologies well understood by the large team? And by beneficiaries? - The impact of the project: impact evaluation is essential for this ACF experience in order to define new interventions in the future. However so far data collection has been very difficult (no well planned from the beginning) and poor. Special effort is expected from the evaluation to value the impact of this intervention. It is expected to analyse the impact of the project according to the following aspects: improved coverage of nutrition program; increased household dietary diversity (FFV distribution), contribution to decreasing and/or preventing global acute malnutrition rates; impact in health education; improved market supply; improved business of project vendors, community perceptions of ACF, and negative effects (specially in the insecure context of some intervention areas in the city). The project objective (lack of micronutrients) and target groups, the value of voucher and project monitoring remain key issues to be dealt with by the evaluation. Amended evaluation criteria as (appropriateness in terms of targeting, project coverage, project implementation process, monitoring and reporting; impact, coherence, accountability and sustainability) are to be added to Annex 2. VII. METHODOLOGY OF THE EVALUATION VII.1 Briefing Prior to the evaluation taking place, a Briefing will be conducted by phone with ACF-UK, lively with ACF France HQ and with the Mission. VII.2 Field activities Data collection by three means: Direct information: Interviews with beneficiaries - Visit to project sites and to the facilities provided to the beneficiaries Indirect information: Interviews with local representatives; interviews with project staff expatriate and national staff); meeting with local authorities, groups of beneficiaries, humanitarian agencies, donor representatives and other stakeholders. > For indirect data collection, standard and participatory evaluation methods are expected to be used (HH interviews and FGDs with beneficiaries, non-beneficiaries, key informants health workers, teachers and leaders) Secondary information analysis e.g. analysis of project monitoring data or of any other relevant statistical data. VII.3 Report The report should be presented in the format set out in Annex 3. The draft report must be submitted no later than 10 days before the end date agreed by the contract. The final report will be submitted no later than the end date of the consultancy contract. A maximum of 5 days is allowed between reception of the draft report and completion of the final report including all feedback. ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

8 The report can only have a maximum of 2 versions (Draft report 1 Draft report 2 (This version includes all the feedback - This version should be the final report but is sent for approval to HQ) and Final report (HQ approval). The evaluation report shall have a maximum length of 50 pages including the Executive Summary at the beginning of the document, Findings, Conclusions and Recommendations. The report should be presented in draft form for comment, before the final report is completed. Relevant comments from the Mission and ACF HQ debriefings should be incorporated in the final report. The final report will be submitted in an electronic version to the given reference staff supporting the programme evaluation (as agreed at the Briefing), including all annexes, together with a hard copy. Annexes to the report will be accepted in the working language of the country and programme subject to the evaluation. VII.4 Debriefing / Learning Workshop The evaluator should facilitate a learning workshop with the following objectives: To present the draft findings of the review to the Mission and other stakeholders To gather feedback on the findings and build consensus on recommendations To develop action-oriented workshop statements on lessons learned and proposed improvements for the future. VII.5 Debriefing with ACF HQ The evaluator should provide a presentation to the ACF HQ on the main findings, conclusions and recommendations of the evaluation, and relevant comments should be incorporated in the final report. VIII. EXPECTED OUTCOME OF THE EVALUATION * The result of this evaluation should be presented in a written report and through several oral presentations, One on the mission, including facilitation of a learning workshop One at the headquarters IX. P ROFILE OF THE EVALUATOR Knowledge in food security programs in emergency situations (both food aid and cashbased interventions) Previous field experience in the evaluation of humanitarian / development projects Relevant degree / equivalent experience related to the evaluation to be undertaken Significant experience in coordination, design, implementation, monitoring and evaluation of programmes Good communications skills and experience of workshop facilitation Ability to write clear and useful reports (may be required to produce examples of previous work) Fluent in French and English Understanding of donor requirements ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

9 Ability to manage the available time and resources and to work to tight deadlines Independence from the parties involved Available for the whole time of the evaluation Conditions apply 40% of fees will be paid after reception of the draft report, 60% will be paid after validation of the final report by ACF-UK Travel to mission and to base and field will be arranged by ACF Accommodation (hotel for briefing / in-country), food, will be at the charge of the evaluator Insurance costs will not be covered and the evaluator shall manage their own insurance and provide details of this cover to ACF-UK before departure Any materials to carry out the evaluation e.g. laptop, must be provided by the consultant X. RIGHTS The ownership of the draft and final documentation belong to the agency and the funding donor exclusively. The document, or publication related to it, will not be shared with anybody except ACF before the delivery by ACF of the final document to the donor. ACF is to be the main addressee of the evaluation and its results might impact on both operational and technical strategies. This being said, ACF is likely to share the results of the evaluation with the following groups: Donor(s) Governmental partners Various co-ordination bodies Intellectual Property Rights All documentation related to the Assignment (whether or not in the course of your duties) shall remain the sole and exclusive property of the Charity XI. L IST OF ATTACHED DOCUMENTS 1. ACF-IN Evaluation Policy & Guidelines 2. ACF-UK Welcome Pack for Consultants * denominates information for the advertisement (consultant recruitment) ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

10 ANNEX 1 BACKGROUND * An earthquake measuring 7.3 on the Richter scale that shook Haiti on January 12th 2010 at has severely affected the departments of Artibonite, the West, the South East, Nippes and the South of Haiti. The epicentre of the earthquake has been identified as being 6 miles deep and 10 miles from the capital. Several subsequent quakes have followed, some with a magnitude of more than 5.5. There is no doubt that millions of people have been affected. ACF in Country Name Brief history of ACF presence in the region / country. Action Against Hunger UK is part of the Action Contre la Faim International Network (ACF). ACF is a non-profit, non governmental organization which mission is to fight hunger, throughout the world. The organization has a solid reputation in the treatment of malnutrition and in the fields of food security, food assistance, water, sanitation, hygiene and health. ACF has been working in Haiti since 1985 and has developed a long-term project strategy in the Water, Sanitation, Hygiene (WASH), Food Security and Nutrition sectors based on capacity building of local authorities. Furthermore, ACF has also launched disaster risk reduction activities. ACF has a permanent presence in Port-au-Prince, Gonaives and Portde-Paix to implement development type projects and has maintained an emergency capacity through the pre-positioning of contingency stocks in both locations and a human resources network of trained national staff readily available through the country. Needs assessment summary Initial reports talk of extremely high numbers of victims (the Haitian government is talking about 111,000 dead; more than 200,000 wounded and 1 to 2,000,000 homeless people) although an exact number is not yet available. The presidential palace and several key government buildings have suffered serious damages. Several United Nations buildings have crumbled, hospitals and health centres have been damaged and some are no longer operating. Those that are able to continue providing healthcare do not have the capacity required to cope with all the wounded. Surrounding hospitals and health centres are also reaching maximum capacity. The airport is only partially operational. The road that links Santo Domingo (the capital of the Dominican Republic) to Port-au-Prince is reported to be passable. The same applies to the road linking Gonaives to Port-au-Prince. The inhabitants are in the street and makeshift camps are being set up in all open spaces: public places, stadiums, churches... Populations outside of Port-au-Prince have also been affected by the earthquake. At present their situation is insufficiently known; the needs in isolated villages and in the urban zones to the South of the island will be identified over the coming weeks. Lastly, the massive displacement of the population to join host families in the less-affected zones will create pressure on already unsatisfactory water resources and the sanitation situation throughout the country. Justification for intervention Water, sanitation and hygiene problems It is thought that the water and sanitation networks have been largely destroyed, and that capacities for producing drinking water have also been severely impacted. The consumable water network was already insufficient for the whole population with some zones relying on damaged wells. Access to water has therefore been largely reduced for a very large part of ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

11 the population. The town s sanitation conditions, that was already very bad prior to the catastrophe, is one of the major risks. The management of human and solid waste, rubble and the corpses are the problems that have been identified. It is also very likely that the victims have lost all of their possessions with their houses, and that they will lack essential items during the first weeks following the catastrophe. Victims are both unable to return to their homes and are also frightened to do so and so are consequently defecating outside giving rise to the risk of transmission of serious diarrheal diseases. The gathering points in open spaces, far from buildings, are increasing the already very high population density. The management of human and solid waste, rubble and the corpses are the problems that have been identified. The configuration on the ground, the lack of space and the city s anarchical urbanisation, largely covered by concrete and tarmac, will challenge the reconstruction of health structures and limit possibilities to dig latrines. It is also expected that the pits of existing latrines are cracked and that faecal pollution may be released into the water tables, particularly in the lower part of the city where the shanty town populations that rely on very shallow wells are concentrated. Temporary latrines will need to be installed very quickly, with pits that are not buried underground (e.g. metal tanks), and that will require regular emptying, for example using a system of rotating tractors, while better adapted latrines are being built. The displacement of victims who are unable to return to damaged buildings and use their commodities leads to the development of spontaneous or more or less organised camps both in the capital and the surrounding areas. The basic needs for such camps must be covered immediately. Populations further from Port-au-Prince (in Carrefour, Leogane, Jacmel) have also been affected by the earthquake. ACF had the opportunity to do a quick assessment in Leogane on January 18 th and estimates that 80 to 90% of the building are damaged or destroyed. The road to Jacmel is still not usable but UN agencies who went there for a quick assessment with an helicopter talked about similar rate of destruction. Lastly, the massive displacement of the population to join host families in the less-affected zones will create pressure on already unsatisfactory water resources and the sanitation situation throughout the country. It is also very likely that many victims have lost all their possessions with their homes, and that they will lack essential items during the first weeks following the catastrophe. Soap and hygiene products will be difficult to access and will not be a priority for the victims. As an example, on 14/01/2010, the day following the earthquake, the media reported that the price of a water berlingot had tripled. This water is sold after purification by reverse osmosis, the facilities for which can no longer function without certain infrastructure being reinstated, particularly energy. Problems related to livelihood, basic food and non food commodities Numerous buildings, homes and facilities have been destroyed or severely damaged and a large part of the population is without shelter having lost the majority of their positions and their sources of revenue. Shops and markets are currently closed, and it is likely that a large part of the city s food stocks have been lost. Even though it is expected that shops will re-open shortly, and that they will be slowly restocked, a large increase in the prices of food products and basic nonfood items is expected. Generalised distributions of food rations by the WFP and its partners are being prepared, but given the forecast difficulties in supply, and those expected in the distributions themselves in urban areas, it is likely that these distributions may take some time to organise and reach the entire affected population. ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

12 The risks of malnutrition for children < 6 months ACF conducted a nutritional survey in Port au Prince in March 2009 that shows that the prevalence of global and severe acute malnutrition remains relatively low and well below the 10% alert level defined by the WHO. The survey also demonstrates that exclusive breastfeeding was not practiced for nearly three quarters of children in this age group. This is confirmed by the national survey EMMUS IV. A psycho-social survey conducted in November 2008 by ACF also showed that women stopped breastfeeding as soon as they experienced strong emotion, or fell ill and that they did not begin breastfeeding again once it had been stopped for several days (experienced in Indonesia s earthquake in 2006). The children themselves may also have suffered intense fear or traumatism which may manifest itself in eating difficulties. This was observed following the cyclones that hit Gonaives in It should therefore be expected that maternal breastfeeding is even less likely to be assured than it was prior to the earthquake, and that this is likely to place children s health at risk. The considerable psychological traumatism of the population The loss of close relatives and friends in terrifying conditions, people/family members trapped under the rubble, worry for those whose situation is still unknown, the sight and smell of numerous corpses, serious injury and loss of all material possessions. To respond to the scale of the crisis and the fact that the mission was directly affected, an immediate international support team was sent within 24 hours comprising technical and support experts. Emergency materials were also dispatched within 48 hours of the earthquake. Current context of intervention We are currently in a transition phase between emergency and post-emergency. People in camps are starting to get out finding more sustainable ways of living. Some camps are also trying to be relocated. The other issues are the movement between rural and urban areas: the urban population that has escaped Port au Prince after the earthquake came back to take advantage of the humanitarian aid as well as some rural population. These population movements need to be assessed to get a real overview of the population needs. Available documents: External documents: - Emergency Food Security Assessment after the earthquake (EFSA), March Emergency Market and Mapping Analysis (EMMA), April 2010 ACF documents: - Programme proposal - Activity Progress Report (March - June 2010) - Capitalization reports (BSF and fresh food vouchers) - ECHO Intermediary Report - Monitoring tools - Market analysis of the 4 markets where the fresh food vouchers programme intervenes ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

13 ANNEX 2 GUIDELINES TO THE EVALUATION CRITERIA Please amend the criteria accordingly if the evaluation is to look more deeply into some of the criteria than others. Impact Impact looks at the wider effects of the action. Impact can be short or long-term, intended or unintended, positive or negative, macro (sector) or micro (household). What direct and indirect evidence is available that the action taken contributed to the improvement of the food security and livelihood of the population? Have the stated project goal, specific objectives, and indicators as shown in the project logical framework been achieved, at least partly? What are the positive and negative, including unexpected, impacts? What is the additional value of ACF approach on the impact of the project? Does it, as aimed, increase the access and acceptance of the programme? Is the outcome satisfying? Are the systems or indicators used to evaluate the impact of the work adequate? Is there any way to improve the impact and its measurement? What is the long-term impact of the program for the individual (household s level)? Does the project (FFV component) contribute to increasing household dietary diversity? Is the value of the voucher well calculated to contribute to dietary diversity? How was the program perceived by non-beneficiaries and local stake-holders? Has BSF strategy contribute to decreasing malnutrition rates? What has been the impact of health education? What has it been the impact in terms of market supply? And improvement of project vendors business? How does the community perceived ACF? What are the adverse effects of the project? Is voucher abuse (using vouchers for non-fresh food) inevitable in an urban context? What means can be developed to quantify this abuse? Is voucher fraud inevitable in an urban context? Does the time required to deal with these overall minor abuse (1 to 2 percent of overall volume, in terms of fake vouchers) invalidate its relevance? Coverage Was there an appropriate knowledge of the needs level? What could be done to improve it? How were the different program components (BSF and FFV) selected? Was there an appropriate or sufficient geographical coverage? Was the targeted population properly covered? Was there a fair identification and targeting of beneficiaries? Are the criteria for beneficiary identification in the different components of the project adequate and pertinent? Both socio economics as well as age and gender should be considered. Were the population and stakeholders made aware of the activities and were they able to access them? ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

14 Does the context of Port au Prince after the earthquake meet the ACF guidelines for appropriateness? Were the items and value of voucher well determined? Was the duration of the assistance the adequate for the stage of emergency answer? Coherence What steps were taken by ACF to ensure the integration of the different project components? How is the FS project coordinated with other technical departments (NUT and WASH)? And with national/international agencies and bodies? How has ACF adapted its strategy considering the arriving of huge number of others NGO in ACF intervention zone and field? What is the coherence of ACF project with national policy throughout the different phases? Is there coherence between donors and partners with ACF project response? Were enough the efforts into formal coordination? Appropriateness/Relevance Relevance is concerned with assessing whether the action correctly identified problems and real needs and whether the projects funded under the Global Plan/Operation were in line with local needs and priorities as well as with donor policy. Besides this general definition, these key questions have to be answered: Operational Strategy & context Does the current project design fit with the specific context of Port au Prince. More specifically, are the risks and assumptions properly defined? Is the implementation strategy adapted to the local community organisation, local customs and culture? Were beneficiaries and or local stakeholders involved in the problem and solution analysis? ACF has work with the community for the ranking of the need. Did the population understand well importance of the ranking? Did ACF explain to the population their participation/involvement for the success of the program? According to that what was their motivation? Are the different activities proposed to filled the problem has been agreed by the communities? Evaluate the degree of satisfaction with the program in the targeted communities. What is the perception of ACF from non beneficiaries about the program? Were mass distributions of specific nutritional products (CSB-oil mix, supplementary plumpy) relevant? What was the and acceptance level for nutritional products by target population (beneficiaries perception of the nutritional products taste, innovation in the way of cooking the CSB-oil mix )? Was the mix of cash-based interventions and classical food-aid coherent? Would it inbetween activities such as fairs be more fitting to the current Port-au-Prince context? What are the recommendations for new programs to complement the fresh food vouchers programme (sellers enhancement, market infrastructures improvement)? Programme Objectives versus needs Is the problematic analyses filled with the context? ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

15 Logic of intervention (refer to the LFA): Are the proposed objectives / results /activities relevant to the needs assessment analysis. Was the program able to meet its objectives, given the lack of assistance for basic food stuff - such as rice, beans, oil, charcoal? What was the impact on respect of program rules (no change given, only fresh products - thus only fresh food merchants - eligible for the program)? Were the sensitisation methods appropriate? Sufficient? Could they be better adapted to the context? Technical Approach Are the activities implementation approach was appropriate to reach the general objective? What other approaches can be developed? Are the proposed activities relevant to improve the food security and livelihood of the population? What are the challenges to ensure the proper implementation of the project? Are there recommendations and practical tools to better implement the project? Was enough integrated ACF knowledge in nutrition issues to implement and monitor the project? What were the impacts to have different BSF and FFV distribution sites, different strict BSF selection criteria for people, and the general coverage of HH in a smaller number of sites for FFV? Could technological advancement such as Mobile Banking be integrated into the cash based interventions in Haiti? Effectiveness Effectiveness measures the extent to which the activities funded under the action achieve their purpose, or whether this can be expected to happen on the basis of the results. Implicit within the effectiveness criterion is timeliness. Effectiveness should indicate the real difference made in practice by the activities funded, how timely the intervention is; equally how far means were used to their maximum effect, how far the intended beneficiaries really benefited from the products or services it made available. Similarly, issues of sourcing and preparedness may be addressed. Are the delivered services to the beneficiaries in accordance with the planned ones (results and indicators)? Compare actual schedules and action plans. Are the planned action plans in accordance with realized ones, and if not why? Are these deviations justified? How this could be improved for the future intervention? Were the different adaptations proposed throughout the programme decided on time? Were these adaptations effectively implemented? What have been the effects? How can ACF improve the performance of project activities? In terms of targeting, what were the advantages of the targeting methodology through committees? And for vendor selection? Regarding project coverage, was it coherent to have different groups of beneficiaries for BSF and FFV? Was it effective the card distribution process in place? And the BSF together with FFV distributions? How has it worked the vendor payment process (voucher collection, internal financial procedures and cash transfer to vendor? Were there any problems with correct identification of vendors? And security problems? ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

16 Was the health promotion activities adapted to the context? And taken in account by beneficiaries? Why was not made a base line? Is it always the case in emergency response interventions? Were the monitoring activities put in place on time? For FFV, was the market price survey sufficient as monitoring tool? Were any other data available (external, secondary resources) for monitoring? Efficiency Efficiency measures how well the various activities transformed the available resources into the intended results (outputs), maximising quality, quantity and timeliness. This links with the question were things done in the best way possible? and thereby also addresses the concept value-for-money that is whether similar results could have been achieved more by other means at lower cost in the same time. General efficiency: Are the tools adapted and efficient? Do the teams understand them? Are the means relevant in regard to the activities proposed? Is the partnership with the local committees efficient? How to improve it? Cost efficiency: Was there any way to reduce the cost per beneficiary but keeping the same level of outcomes? E.g.: Is the number of staff working in the project justified for the number of beneficiaries? Was there any grey area in the reported and effective use of the food security supply dedicated to the beneficiary? What systems were in place to ensure inputs provided are of the highest possible quality and are acceptable to beneficiaries? Was the budget adequate? Was it used appropriately / as intended/ in accordance with original narrative and budget. Could the cost of operations be deemed efficient and line with expected outputs? Was there any overlapping on human resources in the food security department? How to better optimize the use of resources? What systems of financial and logistical monitoring / control were in place in relation to standard procedures? What were the incentives to distribution workers? Where they appropriate and effective? Sustainability Sustainability is concerned with measuring whether an activity is likely to continue after donor funding has been withdrawn and also whether its longer-term impact on the wider development process can also be sustained at the level of the sector, region or country. In the case of strictly humanitarian actions, connectedness might replace the concept of sustainability. It is defined as the extent to which activities of a short-term emergency nature, which are not in principle supposed to be sustainable, are carried out in a context that takes longer-term and interconnected problems into account. An analysis of sustainability will focus on the aspects below. Their relative importance will depend on the nature of the project. ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

17 Beside this general definition, these keys questions have to be taken into account: Was the assistance provided in a way that took into account the longer-term context? Did a rehabilitation or exit strategy exist in any form; was this utilized and adhered to? How successful were the activities to strengthen the capacity of local agencies, organizations and individuals? Was it seen the collaboration with local committees as a capacity building strategy?: How did the community approach contribute to governance? Monitoring Which tools are developed to monitor the progress and impact of the project? Are they implemented effectively? How can they be improved? Were appropriate and relevant indicators developed for this? How is the monitoring process formalized? Is it adapted? Is the information delivered in time (monitoring data / financial follow up) in order to facilitate decision-making? If not what are the constraints and how could they be released? How have lessons learned from monitoring / evaluation been incorporated into the program, and shared with partners? Was there a mechanism which dealt with complaints / dissatisfaction in relation to the program? Are outcomes used to improve the quality of the programme? Gender Include an analysis of the integration and participation of men and women in the project. Is gender considered in ACF s need assessment, and in the implementation of the program? Did the program include special components for women and if so, were these systematically monitored? Is the program team balanced in term of men and women? ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

18 ANNEX 3 REPORT FORMAT The report shall follow the below format. Cover page Title of the evaluation report: Date of the evaluation; Name of the consultant; Indication that the report has been financed by ; the comments contained herein reflect the opinions of the Evaluator only Table of contents Executive Summary A tightly-drafted, to-the-point and free-standing Executive Summary is an essential component. It should be short, no more than two or three pages. It should focus on the key purpose or issues of the evaluation, outline the main points of the analysis, and clearly indicate the main conclusions, lessons learned and specific recommendations. Main body of the report The main body of the report shall elaborate the points listed in the Executive Summary. It will include references to the methodology used for the evaluation and the context of the action. In particular, for each key conclusion there should be a corresponding recommendation. Recommendations should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it both locally and in the Commission Annexes: Listed and correctly numbered. Format for the main body of the report is: 1. Background information Target Area Main objectives of the project Development of the project Describe briefly the development of the project s activities, including the contextual evolution if necessary. 2. Methodology What evaluation method was chosen and why? What are the main constraints (staff, political, security, access, rainy season etc.)? Include here a reference to an appendix such as questionnaire samples of the methods used. Who, when, why and where was interviewed people or sites selected. 3. Findings and discussion Summarise findings under headings use the criteria headings 3.1 Impact 3.2 Coverage 3.3 Coherence 3.4 Relevance / appropriateness 3.5 Effectiveness 3.6 Efficiency ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August

19 3.7 Sustainability 3.8 Monitoring 3.9 Cross-Cutting Issues Wherever possible, use maps, tables, diagrams, and interpretation of qualitative findings Include examples of what people actually say in the interviews Briefly describe the methods used to analyse the information (statistics, PCA, etc.) 4. Conclusions Summarise the main conclusions for each of the sections outlined above, and the main lessons learned and more generally for the typology of this response 5. Recommendations What are the main recommendations to be considered for future interventions according to the evaluation? Where appropriate, make recommendations which relate to each of the sections highlighted above. Annexes Annexes to the report should include, The Evaluation budget Example questionnaires / interview sheets Feedback / outputs from the Debriefings / Workshop ToR ACF Evaluation Food Security Program Emergency Earthquake Haiti August