CALL FOR EXPRESSIONS OF INTEREST Independent Final Evaluation of ACF s Project

Transcription

1 CALL FOR EXPRESSIONS OF INTEREST Independent Final Evaluation of ACF s Project Integrated Emergency Nutrition, Health, Water, Sanitation, Hygiene and Food Security Interventions for conflict affected populations in Yemen 1 Project background The escalation of conflict since the start of the airstrikes in late March 2015 have resulted in population displacements that worsens the vulnerability of the population most affected by chronic poverty and lack of basic services. The population worst affected by the escalating conflict were women, girls and boys who cumulatively made up more than 65% of the entire population. This project is part of an integrated response for conflict affected population in North and South Yemen covering nutrition, health, food security and livelihoods and water, sanitation and hygiene (WASH) interventions. This action aims at reducing the risk of morbidity and mortality associated with acute malnutrition among conflict affected population in Yemen. The general objective of the project is to contribute to reducing the risk of morbidity and mortality associated with acute malnutrition among vulnerable and conflict affected population in Yemen. The specific objective is to preserve lives of vulnerable populations affected by acute malnutrition, related food insecurity or conflicts through integrated support. Specifically the project aims to achieve the following outputs/results: - Nutrition status of women, boys and girls improved through preventative and curative nutritional program. - Targeted internally displaced persons (IDP) and host communities show an improvement in good hygiene practices and have improved access to water and sanitary facilities through emergency WASH minimum package. - Access to and availability of essential preventive, promotive and curative primary health care services for boys, girls and pregnant lactating women is improved. - Food insecure male and female headed IDP and host community households have improved food access through food assistance activities. The project timeframe is from July 2015 to June 2016 and has been funded by ECHO ( 1,300,000). 2 Scope of the evaluation The overall purpose of the evaluation is to assess the overall performance of the project, analyzing if the implementation of activities led to the intended outputs and if the latter (and the reasons why) have contributed to preserved lives of vulnerable populations affected by acute malnutrition related food insecurity or conflicts. The evaluation will use the OECD DAC criteria approach to evaluate the whole project in the two governorates of Hodeida and Hajjah. It will cover all components of the intervention but should focus in more depth in WASH and nutrition components. It will also cover all selected target groups of 1

2 beneficiaries. Finally, it is also intended to provide operational and strategic recommendations for the implementation in the next phases of projects and for ACF more generally. It will require the following deliverables: 1. Inception Report outlining the evaluation methodology to be used by the evaluator and a final work plan; 2. Stakeholder workshop: The evaluator shall facilitate a learning workshop in country to present the draft report and the findings of the evaluation to the project and key stakeholders; to gather feedback on the findings and build consensus on recommendations; 3. Draft evaluation report with specific recommendations; lessons learnt and good practices; 4. Final evaluation report (max. 30 pages) incl. executive summary. Gender equality issues shall be explicitly addressed throughout the evaluation activities of the consultant and all outputs including final reports or events need to be gender mainstreamed. 3 Evaluation arrangements and professional requirements An Independent Evaluator will conduct a final independent evaluation from June 6 th, 2016 to July 20 th, The assignment is for approximately 26 work-days, including approximately 13 days in the field. The assignment will require travel to Yemen. The field office in the country to be visited and A C F project staff will help in organizing logistics. Due to the changing context, previous approval from ECHO, the evaluation could be cancelled before or during field work. Please see the Terms of Reference for further information on security issues. Selection of the Independent Evaluator will be done by the Evaluation Learning and Accountability team and will be based on the strength of the qualifications provided by potential candidates through their expressions of interest for the assignment. Interested candidates should include in the expression of interest: 1. An updated CV; 2. A detailed presentation of their background and experience, highlighting significant field experience in the evaluation of humanitarian / development projects relevant to this assignment; 3. A statement of availability for the entire duration of the assignment; 4. A daily consulting fee in EUR; 5. A couple of work samples. 6. At least two references. Some of the qualifications applicants should have: A generalist with experience and knowledge in integrated emergency programs covering nutrition, WASH and livelihood interventions; Relevant degree / equivalent experience related to the evaluation to be undertaken; Significant experience in coordination, design, implementation, monitoring and evaluation; Ability to manage the available time and resources and to work to tight deadlines; Independence from the parties involved; Excellent written and oral communication skills in English. Local languages are an advantage. The deadline to submit expressions of interest for the evaluation is by close of business on May 11 st, Applicants should submit a complete expression of interest (including all six elements highlighted above) via to ela@actionagainsthunger.org.uk and indicate: Yemen - Emergency in the s subject line. 2

3 TERMS OF REFERENCE For the Independent Final Evaluation of ACF s Integrated Emergency Nutrition, Health, Water, Sanitation, Hygiene and Food Security Interventions for conflict affected populations in Yemen Project funded by: ECHO Summary Table Project Name Contract Number Sector Partners (if applicable) Location Duration Starting Date 01/07/2015 Ending Date 30/06/2016 Project Language English Donor & Contribution/s ECHO ( 1,300,000) Country Office administering ACF Yemen the Project Responsible ACF HQ ACF France Evaluation Type Final Independent Evaluation Dates 06/06/ /07/2016 Integrated Emergency Nutrition, Health, Water, Sanitation, Hygiene and Food Security Interventions for conflict affected populations in Yemen YEA1D Multisector (Nutrition, Health, WaSH, Food Security) No Hodeidah, Hajjah, Lahj and Abyan - Governorates affected by armed conflict or high levels of malnutrition 12 months 1

4 List of Acronyms ACF: Action Contre la Faim ANC: Antenatal Care ECHO: European Commission's Humanitarian aid and Civil Protection Office FS: Food Security GARWSP: General Authority for Rural Water Supply IDP: Internally Displaced Persons KAP: Knowledge, Attitudes and Practices MAM: Moderate Acute Malnutrition MoPHP: Ministry of Public Health & Population PDM: Post Distribution Monitoring PLW: Pregnant and Lactating Women PNC: Postnatal care SAM: Severe Acute Malnutrition SQUEAC: Semi Quantitative evaluation of access and coverage SMART: Standardized Monitoring and Assessment of Relief and Transitions YHRP: Yemen Humanitarian Response Plan U5: Under 5 WASH: Water, Sanitation and Hygiene 2

5 1. PROJECT BACKGROUND 1.1. Map of intervention area: Hodeidah and Hajja Governorates 1.2. Rational for the Project Armed conflict spread rapidly across much of Yemen since mid-march 2015, with devastating consequences for civilians. By then violence killed over 2,800 people and close to 13,000 injured were reported in hospitals. The United Nations Office of the High Commissioner for Human Rights identified over 1,400 civilians killed and over 3,400 civilians injured. According to the updated Humanitarian Needs Overview, 21 million people - 80 per cent of the population - required some form of humanitarian protection or assistance. This represented a 33 per cent increase since December Besides, according to this report, it seemed that 12.3 million Yemenis suffered from food insecurity. The Integrated Food Security Phase Classification rated 10 out of 22 governorates at the emergency level - one level short of "famine" as of 22nd of June. Access to safe drinking water and proper sanitation was also poor with over 20.4 million people affected. Malnutrition situation was also critical with an estimation of 1.5 million children under 5 (U5) and pregnant and/or lactating women (PLW) requiring services to treat or prevent acute malnutrition. Action contre la Faim (ACF) had been running emergency life-saving interventions in Hodeidah and Hajjah since 2012, observing deterioration in the context following the outbreak of conflict across the country. In the north, the airstrikes and the fighting especially in Sa'ada resulted in population displacement. About 300,000 IDPs and conflict affected host households were registered in Hodeidah, Hajjah and Lahj, Governorates where ACF was implementing integrated nutrition, health, water sanitation and hygiene (WASH) and Food Security (FS) interventions. This resulted in population displacements worsening the vulnerability of the population most affected by chronic poverty and lack of basic services. The population worst affected by the escalating conflict were women, girls and boys who cumulatively made up more than 65% of the entire population. 3

6 This project was part of an integrated response for conflict affected population in North and South Yemen covering nutritional, health, FS and WASH interventions. The action aimed at reducing the risk of morbidity and mortality associated with acute malnutrition among conflict affected population in Yemen Project description General Objective: Contribute to reducing the risk of morbidity and mortality associated with acute malnutrition among vulnerable and conflict affected population in Yemen. Specific Objective: Preserving lives of vulnerable populations affected by acute malnutrition, related food insecurity or conflicts through integrated support. Expected Results: ER1: Nutrition status of women, boys and girls improved through preventative and curative nutritional programme ER2: Targeted IDP and host communities show an improvement in good hygiene practices and have improved access to water and sanitary facilities through emergency WASH minimum package. ER3: Access to and availability of essential preventive, promotive and curative primary health care services for boys, girls and PLW is improved ER4: Food insecure male and female headed IDP and host community households have improved food access through food assistance activities. The Project logframe is attached in Annex I. Within the 12 months of the action, ACF was expected to target 36,730 beneficiaries, with benefiting from nutritional interventions and further 9,500 reached by health programs. In addition, 8,400 beneficiaries were expected to benefit from food security and livelihoods (FSL) interventions. The WASH interventions were expected to reach 9,800 persons. The target group for these interventions were mainly boys and girls U5 with severe acute malnutrition (SAM) and moderate acute malnutrition (MAM), and PLW. IDP households were also targeted and expected to benefit from FS and WASH interventions. The action was initially designed to be implemented in the three governorates of Hodeida, Hajjah and Lahj. However, in case of access limitations in these governorates (likely in Lahj), ACF was expected to evaluate and after consultations with ECHO field office, move the interventions proposed (or part of them) to other governorates prioritized according to the Humanitarian Response Planning. ACF programmes are integrated into the local and national systems and have a community-centred approach to build local capacity. In terms of nutrition, this action has been addressing emergency needs in remote areas not covered by the Ministry of Public Health and Population s (MoPHP) while ensuring further strengthening of the MoPHP system already weakened by the crisis. Integration was also expected to be built with other local and governmental authorities such as the General Authority for Rural Water Supply (GARWSP). The 2015 Yemen Humanitarian Response Plan (YHRP) and the various clusters have given the highest priority to these critical nutrition, WASH and FSL sectors and activities. ACF works in close collaboration and coordination with other humanitarian partners at all levels in order to avoid duplication and enhance complementarity particularly through active participation in working groups and cluster meetings, and also directly with line ministries such as the MoPHP. The management set up of the project involved ACF France, in charge of the technical support; the Country Office in Yemen, in charge of the donor reporting, the technical and overall coordination; the 4

7 Field Offices in Hodeidah and Aden, in charge of the field coordination and the implementation. The latter involved a project operational team based in Hodeidah and Aden Project Current Status ACF started the implementation of the integrated nutrition, FSL and WASH activities in Hajjah, Hodeidah and Lahj governorates during the first eight months of the project. However, between December and January, the implementation of the action was hindered by field access limitations particularly in Hajjah governorate. Nutrition interventions provided access to quality therapeutic and supplementary feeding in Hodeidah and Hajjah Governorates. As of April 2016, through its mobile activities ACF screened 30,704 children, from whom 3,398 SAM children were treated. A total of 310 U5 MAM children and 256 PLWs were provided supplementary feeding. A total of 27,992 beneficiaries have been reached under the health activities: 7,832 children benefited from medical consultation and treatment, 3,775 PLW received antenatal and postnatal care (ANC/PNC), 1,704 care givers received Infant and young child feeding (IYCF) counselling and 14,681 benefited from health education. FSL interventions are currently being implemented in Aslam District in Hajjah Governorate and Tur Al Baha District in Lahj Governorate. Monthly cash transfers have been distributed to 1,210 households with 370 households in Lahj receiving all three monthly cycles and 840 households in Hajjah having received one cash transfer round out of three. Subsequent rounds will be completed as soon as access permits are provided. In terms of WASH, implementation of activities is on-going in both the northern and southern governorates, and 3,024 persons have been provided with safe water, five wells have been rehabilitated and 4,900 persons provided with hygiene kits. 2. PURPOSE AND OBJECTIVES OF THE EVALUATION 2.1. Rational for the Evaluation This evaluation is conducted as an exercise of accountability towards the donor and the beneficiaries. It is also expected to contribute to a better understanding of the strengths and weaknesses of the intervention, drawing lessons learnt and making operational and strategic recommendations that can be used to improve the implementation of a potential next phase or similar interventions in the future Objectives of the Evaluation 1. To assess if the implementation of the activities have lead to the expected results 2. To asses if the results/outputs have lead to preserve lives of vulnerable populations affected by acute malnutrition related food insecurity or conflicts 3. To assess if there is evidence to believe that the preservation of the lives of targeted affected population has contributed to reduce their risk of morbidity and mortality associated with acute malnutrition 2.3. Users of the Evaluation Direct users: ACF ACF France Field Level Country Director, Deputy Country Director, Heads of Department (Nutrition & Health, Food security and WASH) 5

8 Indirect users: ACF Other ACF International Network Donors, UN agencies and Global Clusters, NGOs and NGO Consortiums, etc Use of the Evaluation To learn from experience to develop new strategies, to collect leassons learnt and good practices for future projects in country, as well as to collect evidence of ACF success on the ground. It is also intended to provide operational and strategic recommendations for the implementation in the next phases of projects and for ACF more generally. 3. EVALUATION SCOPE The evaluation is expected to cover the whole project (all objectives) and the two governorates of Hodeida and Hajjah. It will cover all components of the intervention (nutrition, health, FSL and WASH) but should focus in more depth in WASH and nutrition interventions Elements not covered by the evaluation The evaluation will not cover the Southern governorates and geographical locations of Lahj and Abhyan due to logistical and security contraints Cross-cutting issues The evaluation is expected to address gender issues in line with ACF gender policy and toolkit as well as the degree of community participation. 4. EVALUATION APPROACH As per ACF Evaluation Policy and Guidelines 1, ACF adheres to the Organisation for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) criteria approach for evaluating its programmes and projects. Specifically, ACF uses the following criteria: Design 2, Coherence, Coverage, Relevance/Appropriateness, Effectiveness, Efficiency, Sustainability and Likelihood of Impact 3. Evaluation questions have been developed to help the evaluator assess the project against these criteria (refer to Annex II). The evaluator may adapt the evaluation criteria and questions, but any fundamental changes should be agreed between the ELA at ACF-UK and the evaluator and reflected in the inception report. All independent external evaluations are expected to use DAC criteria in data analysis and reporting. In particular, the evaluator must complete the DAC criteria rating table (refer to Annex V) and include it as part of the final evaluation report Design is not a DAC criterion but is has been introduced as an area of performance to be analysed since 2015 in all evaluations terms of reference. This is intended to gather specific and consistent evidence on design aspects to assess related strengths and weaknesses. 3 The impact criterion has been adapted to likelihood of impact to account for the impossibility to assess accurately the attribution of the observed effects to the project when conducting performance evaluations. Impact measurement can only be accurately done with experimental and quasi experimental evaluation designs. Assessing the likelihood of impact we imply the assessment of the contribution of the project to the observed effects rather than the attribution, which is technically more correct. 6

9 5. EVALUATION DESIGN AND METHODOLOGY The space and ability to deliver humanitarian assistance in Yemen is restricted by security-related risks. The risks include kidnappings, attacks on aid workers, refusals to grant visas or local travel permits, red tape and other obstacles imposed by national governments and de-facto authorities in an attempt to regulate humanitarian activities in areas under their control. Internal organizational security rules may also constitute a major impediment in insecure environments. These restrictions do present methodological challenges for external evaluations (particularly where non-nationals are engaged) where interviews and assessments cannot be done in person. For the required quality, scope and depth of this evaluation, the evaluator will need to have access to the field locations. The options include having a hybrid experienced external consultant in partnership with a national/ local consultant for a balance between remote support and access to the field for data collection for consistent and quality data. Other option would be to recruit members of Diaspora communities or experienced local consultants who will not suffer similar restrictions in accessing field locations. This is a one-group design evaluation, without including any comparison with units (people, communities, etc.) that did not participate in the project. This evaluation is also expected to look at units participating before and after the projet has been in operation. Below is outlined the suggested methodological approach for the evaluator to collect quantitative and qualitative data. The evaluator will to the extent possible develop data gathering instruments and methods which allow collecting sex-disaggregated data. The instruments need to make provision for the triangulation of data where possible Evaluation Briefing Prior to the evaluation taking place, the evaluator is expected to attend an evaluation technical briefing with the ELA ACF-UK. Briefings by telephone must be agreed in advance ACF HQ Briefing (in Paris) As part of the evaluation, the evaluator is expected to have a project and security briefing with HQ stakeholders to get preliminary information about the project being evaluated Desk review The evaluator will undertake a desk review of project materials, including the project documents and proposals, progress reports, outputs of the project (such as publications, communication materials, videos, recording etc.), results of any internal planning process and relevant materials from secondary sources. 5.4 Sampling Due to security constraints and challenges in access, a convenience sampling is recommended for this evaluation. The criteria for sampling would be based on access trying to capture to the extent possible best and worse performing areas. 5.5 Inception Report At the end of the desk review period and before the field mission, the evaluator will prepare a brief inception report. The report will be written in english and will include the following sections: Key elements of the TORs to demonstrate that the evaluator will adhere to the TORs; Present the methodological approach to the evaluation (including an evaluation matrix in annex to specify how the evaluator will collect data to answer the evaluation questions) and point out the limitations to the methodology if any and the choice of sites per field visit; 7

10 Provide a detailed evaluation workplan and; State adherence to ACF Evaluation Policy and outline the evaluation report format. Data collection tools in annex. The inception report will be discussed and approved by the ELA in ACF-UK and shared with the field. 5.6 Field Mission Briefing at Field Level As part of the evaluation, the evaluator will firstly have a briefing with the Country Director, Deputy Country Director and the relevant technical focal points (Heads of Nutrition, WASH and FSL departments). Briefings by telephone must be agreed in advance Data Collection Primary data collection techniques The evaluator is expected to use a participatory approach for data collection. The evaluator should interview key project stakeholders (expatriate/national project staff, local/national representatives, local authorities, humanitarian agencies, or donor representatives) as per the list in Annex IV. The evaluator will use the most suitable format for these interviews as detailed in the inception report. The evaluator will also use interviews and focus group discussions to collect information directly from beneficiaries. To the extent possible, towards enriching triangulation, the evaluator is expected to also apply interviews and/or focus group discussions with non-beneficiaries and other key informants. Field visits The evaluator will visit the project sites and the facilities provided to the beneficiaries (if any). Secondary data collection techniques: Desk review The evaluator will further review complementary documents and collect project monitoring data or of any other relevant statistical data including Semi Quantitative evaluation of access and coverage (SQUEAC), Standardized Monitoring and Assessment of Relief and Transitions (SMART) or KAP (Knowledge, Attitudes and Practices) surveys. Debriefing and stakeholders workshop The evaluator shall facilitate a learning workshop in country to present preliminary findings of the evaluation to the project and key stakeholders (including national actors); to gather feedback on the findings and build consensus on recommendations; to develop action-oriented workshop statements on lessons learned and proposed improvements for the future. 5.7 Evaluation Report The evaluation report shall follow the following format and be written in English: Cover Page; Summary Table to follow template provided Table of Contents Executive Summary must be a standalone summary, describing the project, main findings of the evaluation, and conclusions and recommendations. This will be no more than 2 pages in length) Background Information Methodology describe the methodology used, provide evidence of triangulation of data and presents limitations to the methodology Findings includes overall assessment of the project against the evaluation criteria, responds to the evaluation questions, all findings are backed up by evidence, cross-cutting issues are mainstreamed and; unintended and unexpected outcomes are also discussed 8

11 Conclusions are formulated by synthesizing the main findings into statements of merit and worth, judgements are fair, impartial, and consistent with the findings Lessons Learnt and Good Practices presents lessons that can be applied elsewhere to improve programme or project performance, outcome, or impact and; identify good practices: successful practices from those lessons which are worthy of replication; further develop on one specific good practice to be showcased in the template provided in Annex VI Recommendations should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it both locally. They should follow logically from conclusions, lessons learned and good practices. The report must specify who needs to take what action and when. Recommendations need to be presented by order of priority Annexes These should be listed and numbered and must include the following: Good practice template provided in annex VI, Evaluation Criteria Rating Table (annex V), list of documents for the desk review (annex III), list of persons interviewed (annex IV), data collection instrument, evaluation TORs The whole report shall not be longer than 30 pages, 50 pages including annexes. The draft report should be submitted no later than 10 calendar days after departure from the field. The final report will be submitted no later than the end date of the consultancy contract. Annexes to the report will be accepted in the working language of the country and project subject to the evaluation. 5.8 Debriefing with ELA ACF-UK The evaluator should provide a debriefing to the ELA in ACF-UK to discuss any issues related to the evaluation report. 5.9 Debriefing with ACF HQ The evaluator should provide a debriefing with the relevant ACF HQ on her/his draft report, and on the main findings, conclusions and recommendations of the evaluation. Relevant comments should be incorporated in the final report. 6 KEY DELIVERABLES The following are the evaluation outputs the evaluator will submit to the ELA in ACF-UK: Outputs Deadlines Inception Report 11/06/2016 Stakeholders workshop 28/06/2016 Draft Evaluation Report 06/07/2016 Final Evaluation Report 20/07/2016 All outputs must be submitted in English and under Word Document format. The quality of the inception report and the evaluation report will be assessed by the ELA in ACF-UK. The evaluator is expected to follow the format, structure and length as defined under section 5.4 and 5.6 above. 9

12 7 MANAGEMENT ARRANGEMENTS AND WORKPLAN The evaluator will directly report to the ELA in ACF-UK. The evaluator will submit all the evaluation outputs directly and only to the ELA in ACF-UK. The ELA in ACF-UK will do a quality check (ensure required elements are there) and decide whether the report is ready for sharing. The ELA will forward a copy to key stakeholders for comments on factual issues and for clarifications. The ELA will consolidate the comments and send these to the evaluator by date agreed between the ELA and the evaluator or as soon as the comments are received from stakeholders. The evaluator will consider all comments to finalize report and will submit it to the ELA who will then officially forward to relevant stakeholders. Once the evaluation is completed the ELA ACF-UK will prepare the management response follow-up form to track implementation of the recommendations outlined in the evaluation report. A review of the follow-up process will be undertaken six months after the publication of the evaluation report. 10

13 7.6 Tentative Workplan 4 Activities Evaluator Working Days Dates ECHO Validation ToR 27/04/2016 Recruitment - Ad 28/04/ /05/2016 Recruitment - Selection 12/05/ /05/2016 Recruitment Contractual Arrangements 16/05/ /05/2016 Visa Processing (2-3 weeks) 19/05/ /06/2016 Days / Weeks Thu Wed (14 days) Thu Fri Mon Wed (3 days) Thu Fri (2.5 weeks) Evaluation briefing with ACF-UK ELA /06/2016 Mon Briefing with HQ in Paris 2 06/06/2016- Mon Tue 07/06/2016 Desk review, preparation of field work and 3 09/06/ Thu Sat prepare Inception Report 11/06/2016 Review of the Inception Report 13/06/2016 Mon Travel to Yemen 14/06/2016 Tue In country interviews with project staff 1 15/06/2016 Wed Field work, collection and analysis of secondary data & meeting with stakeholders 10 16/06/ /06/2016 Stakeholders Workshop in country 1 28/06/2016 Tue Travel back from the field 29/06/2016 Wed Evaluation debriefing with ACF-UK ELA /06/2016 Thu Evaluation debriefing with HQ /06/2016 Thu Draft Report 5 01/07/ /07/2016 ACF-UK: Quality check and initial review by ELA, 07/07/2016 circulate draft report to key stakeholders, 15/07/2016 consolidate comments of stakeholders and send to evaluator Final report on the basis of stakeholders, Mission, 3 18/07/ HQ, and ACF-UK comments 20/07/2016 Total: 26.5 Thu Mon Fri Wed Thu Fri Mon Wed 7.7 Profile of the evaluator The evaluation will be carried out by an international evaluation consultant with the following profile: A generalist with experience and knowledge in integrated emergency programmes covering nutrition, WASH and livelihood interventions; Significant field experience in the evaluation of humanitarian / development projects; Relevant degree / equivalent experience related to the evaluation to be undertaken; Significant experience in coordination, design, implementation, monitoring and evaluation of programmes; Good communications skills and experience of workshop facilitation; Ability to write clear and useful reports (may be required to produce examples of previous work); 4 Consultants are expected to work 6 days a week (either Sundays/Fridays or whatever day the field office has off will not be paid) during their consultancy contract. Travel days are not paid as they are not working days as such. 11

14 Fluent in english; Understanding of donor requirements; Ability to manage the available time and resources and to work to tight deadlines; Independence from the parties involved. 8 LEGAL AND ETHICAL MATTERS The ownership of the draft and final documentation belong to the agency and the funding donor exclusively. The document, or publication related to it, will not be shared with anybody except ACF before the delivery by ACF of the final document to the donor. ACF is to be the main addressee of the evaluation and its results might impact on both operational and technical strategies. This being said, ACF is likely to share the results of the evaluation with the following groups: Donor(s) Governmental partners Various co-ordination bodies For independent evaluations, it is important that the consultant does not have any links to project management, or any other conflict of interest that would interfere with the independence of the evaluation. 8.6 Intellectual Property Rights All documentation related to the Assignment (whether or not in the course of your duties) shall remain the sole and exclusive property of the Charity. 9 ANNEXES TO THE TORs I. Project Logframe II. Detailed Evaluation Questions III. List of Project documents for the desk review IV. List of people to be interviewed V. Evaluation Criteria Table VI. Good practices Template 12

15 Annex I: Project Logframe Title of the Action General Objective Integrated Emergency Nutrition, Health, Water, Sanitation, Hygiene and Food Security interventions for conflict affected populations in Yemen Contribute to reducing the risk of morbidity and mortality associated with acute malnutrition among conflict affected population in Yemen Intervention Logic Objectively Verifiable Indicators Sources of Verification Risks and Assumptions Specific Objective Result 1 Result 2 Preserving lives of vulnerable populations affected by the conflict through integrated NUT / Health / WASH / FSL support Nutrition status of women, boys and girls improved through curative nutritional programme Targeted IDPS and host communities have improved access to water and sanitary 75% of severe acutely malnourished U5 boys and girls enrolled is discharged as cured 5500 boys, girls and women received medical consultation and treatment for common illnesses IDPs and host communities with improved access to potable water and sanitary facilities 8,400 people have improved access to diversified food through cash/voucher based interventions 75% Recovery rate 3,800 severely acutely malnourished children and 800 moderately malnourished children admitted to therapeutic treatment programmes 9,800 persons provided with safe water 60% of persons able to mention the key times for hand-washing and the routes in feco-oral transmission ACF registration records; ACF TFP database; MoH reports Register and database; Medical Records Consultation cards and stock register Water Quality Follow-up; Distribution and household visits reports ACF Baseline & end line survey report; Food Consumption Score (FCS) ACF Baseline & end line survey report; Food Consumption Score (FCS) and Household Dietary Diversity Score (HDDS) TFP database Screening and referral database; Activity Progress Report Distribution and household visits reports Water quality follow-up database KAP survey Principal and sub-agreements with the government authorities remain valid No unpredicted, forced or intentional suspension of program or any other reasons affecting negatively on ACF Charter Access to most vulnerable areas and population is ensured in Hodeida and Hajjah Governorates Access to the most vulnerable beneficiaries is not hindered in Lahj Governorate Access to therapeutic products needed for treatment of malnutrition from UNICEF is gained Fuel is available and transporters accept to deliver goods in the areas of intervention Markets are well provided with food and prices remain acceptable in areas where cash/voucher intervention is implemented. Boat shipping from Djibouti remains functional 13

16 Result 3 Result 4 Activities R1 Activities R2 Activities R3 Activities R4 facilities and show an improvement in hygiene practices through emergency WASH minimum package. Access to and availability of essential preventive, promotive and curative primary health care services for boys, girls and pregnant/lactating women is improved Food insecure IDPs and host communities households have improved food access through food assistance activities 6,000 persons provided with emergency toilet/latrine 5500 of children (boys & girls) received consultation and treatment for common illnesses through mobiles clinics PLWs received preventive and curative consultation through mobile clinics 80% of pregnant women attending ANC 1 90% of the target population achieves borderline 5 Food Consumption Score (FCS) at minimum The mean Household Dietary Diversity Score (HDDS) of target population has increased by the Target % over the project period 80 Latrines database Community Leaders are supportive and Community Resource persons are available for training and willing to engage in the program Medical Records; Beneficiary Register; Morbidity reports PLW register; Prescription for common illnesses Stock register for consumption of medicine Baseline and End-line Surveys Market monitoring survey 1.1. Screening of under 5 boys and girls at community and facility levels for detection of Severe Acute Malnutrition cases and referral to Mobile clinics and fixed sites for further treatment 1.2. Health & Nutrition awareness through awareness sessions of PLWs regarding nutrition and health & hygiene at outreach and medical centres 2.1. Provision of emergency water supply 2.2. Community and household-based hygiene promotion sessions 2.3. Provision of emergency sanitary facilities at community level 2.4. Distributions of NFIs/Hygiene Kits 3.1. Ante-natal and post-natal services for pregnant and lactating women 3.2. Treatment of common illnesses among boys, girls and PLWs attending targeted mobile health facilities 4.1. Implementation of monthly resource transfer to targeted food insecure IDPs and host households 5 Can use "borderline" threshold in an acute emergency, in which case the target will be 90%, and the "acceptable" threshold in a milder situation or during a recover phase. 14

17 Annex II: Evaluation Criteria and Detailed Questions To assess the project against each evaluation criteria, the evaluator will respond to the following evaluation questions: Design: A measure of whether the design is logical, allows for RBM and include a sustainability strategy involving local partners and beneficiaries - Degree to which the project has been able to address the issues outlined under Risks and Assumptions in the project document. - Are the monitoring tools appropriate and in line with the challenging Yemeni context and community dynamics? - Is the project approach sound? To advise on possible changes to project approaches for development of future interventions. - Are the different components of the project properly integrated in the project design? Are there mechanisms to enhance integration of project components at the community level? Specific interest in WASH in Nutrition components. - To what extent did the project involve men, women, boys and girls in the design? - Did the project adequately address traversal issues of gender and conflict sensitivities and how can this be enhanced? Relevance/Appropriateness: A measure of whether interventions are in line with local needs and priorities (as well as donor policies) - Was the project implemented taking into account the different needs of men, women, boys and girls? Coherence: A measure of whether interventions are consistent with existing interventions, global and national policies and strategies to ensure consistency, maximize synergies and minimize duplication - To what extent were the project interventions consistent with the existing intervetions? - To what extent were project interventions complementary with the government policies? Coverage: A measure of whether interventions meet the need to reach major population groups facing life threatening suffering wherever they are - Are the quality of services adequate and appropriate in terms of access (coverage), delivery and follow-up of therapeutic feeding programme? - Was the beneficiary targeting effective? What was the degree of inclusion of the most vulnerable groups (e.g. under-5 years children, Female-headed households, etc.)? Efficiency: A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results. - Have the inputs available (funds, staff, procedures, transportation, etc.) been aligned for the proper implementation of activities given the challenging context? Effectiveness: A measure of the extent to which the interventions objectives were achieved, or are expected to be achieved, taking into account their relative importance and illustrating the effectiveness of ACF approach - To what extent the degree of achievement of the outputs/results have determined the achievement of the specific objective? Why or why not? Please provide a table specifying the level of achievement of the logframe indicators targets to assess the aforementioned issue. - Determine how far the project has been able to address the issues of life saving interventions in a challenging fragile context 15

18 - Have the monitoring tools been used adequately within the challenging Yemeni context and community dynamics? - Have the different project s components been implemented in an integrated way? Have the integration been enhanced at the community level? How could integration be further enhanced at the community level? Specific interest in WASH in Nutrition interventions. - To what extent did the project involve communities in the project s implementation and follow-up? Sustainability: A measure of whether the benefits of an activity are likely to continue after donor funding has been withdrawn and project activities officially cease. - Has the exit strategy been successful? Likelihood of impact: Early signs of positive and negative, primary and secondary, short, mid and longterm effects produced by an intervention, directly or indirectly, intended or unintended - Is there evidence to believe that the preservation of the lives of targeted affected population has contributed to reduce their risk of morbidity and mortality associated with acute malnutrition? 16

19 Annex III: List of Project documents for the desk review The following documents 6 will be reviewed by the evaluator during the desk review phase: Document Project Proposal Project Interim Narrative Reports Different sector reports including baseline, PDM and endline survey reports Monthly Activity Progress Reports Description Project proposal document submitted by ACF to ECHO for funding. Progress reports prepared and submitted to ECHO by ACF. Accessment and monitoring reports prepared by the different sectors. Activity progress reports prepared monthly both narrative and quantitative. 6 The list is not exhaustive 17

20 Annex IV: List of people to be interviewed The list will be provided to the selected consultant. 18

21 Annex V: Evaluation Criteria Table The evaluator will be expected to use the following table to rank the performance of the overall intervention using the DAC criteria. The table should be included either in the Executive Summary and the Main Body of the report. Criteria Design Relevance/Appropriateness Coherence Coverage Efficiency Effectiveness Sustainability Likelihood of Impact Rating (1 low, 5 high) Rationale 19

22 Annex VI: Good Practice Template The evaluation is expected to provide one (1) key example of Good Practice from the project. This example should relate to the technical area of intervention, either in terms of processes or systems, and should be potentially applicable to other contexts where ACF operates. This example of Good Practice should be presented in the Executive Summary and the Main Body of the report. Title of Good Practice (Max 30 words) Innovative Features & Key Characteristics (What makes the selected practice different?) Background of Good Practice (What was the rationale behind the good practice? What factors/ideas/developments/events lead to this particular practice being adopted? Why and how was it preferable to other alternatives?) Further explanation of chosen Good Practice (Elaborate on the features of the good practice chosen. How did the practice work in reality? What did it entail? How was it received by the local communities? What were some of its more important/relevant features? What made it unique?) Practical/Specific Recommendations for Roll Out (How can the selected practice be replicated more widely? Can this practice be replicated (in part or in full) by other ACF programmes? What would it take at practical level? What would it take at policy level?) How could the Good Practice be developed further? (Outline what steps should be taken for the practice to be improved and for the mission to further capitalise on this good practice) 20