CALL FOR EXPRESSIONS OF INTEREST Independent Final Evaluation of Action Against Hunger s Project:

Transcription

1 CALL FOR EXPRESSIONS OF INTEREST Independent Final Evaluation of Action Against Hunger s Project: Reinforcing Institutional Capacity for Treatment of Acute Malnutrition, Prevention of Malnutrition and National Sensitization for Nutrition Security in Western Area, Sierra Leone 1 Project background Action Against Hunger is implementing a nutrition security project in Western Areas from 1 January 2016 to 31 December This project is the continuation of the project Action Against Hunger has implemented from September 2013 to October 2015 in Western Area. The areas of interventions have been chosen along with representatives of the Ministry of Health and Sanitation (MoHS), Ministry of Agriculture Forestry and Food Security (MAFFS). The specific objective of the project is to reinforce institutional capacity for quality treatment of acute malnutrition, improve preventive strategies and to raise national awareness on nutritional security in the communities. The aim of the project is to strengthen the capacities of the MoHS at local and national level to ensure quality implementation of the Integrated Management of Acute Malnutrition (IMAM) protocol following its national revision in This project is part of a larger regional intervention in West Africa, co-financed by Agence Française de Développement - French Development Agency (AFD), which concerns three countries for operational implementation of activities (Niger, Burkina Faso and Sierra Leone) and seven countries for advocacy actions (Mauritania, Guinea, Liberia, Ivory Coast, Nigeria, Chad, Mali). The overall goal of the project is to contribute to the improvement of maternal and child nutrition in West Africa. In Sierra Leone this project was co-funded by Irish Aid. The external evaluation will cover the IrishAid funded part of the project Project Objectives The project s principal objective is: To contribute to the improvement of nutritional security of children and mothers in Western Area, Sierra Leone. The specific objective is: Improving the nutritional status of children U5 and PWLM (Pregnant Women and Lactating Mothers) through the integration of nutrition in a strengthened health system and in preventive approaches to the direct causes of under nutrition. 2 Scope of the evaluation The general objective of the evaluation is: to assess the overall performance of the project and to determine if the intervention has reached its intended outputs and objectives. The specific objectives of the evaluation are: a. To assess whether the design of the project is based on beneficiaries needs (sex and age disaggregated), recommendations from previous projects and studies, has a sustainability strategy and allows for Results-Based Monitoring b. To assess whether the project is relevant and appropriate given the local context, culture and needs of the population c. To assess whether the project is aligned with other interventions by Action Against Hunger and with interventions by other actors and to what extent the different technical sectors were integrated in this project d. To assess whether the project was able to reach the most vulnerable groups 1

2 e. How efficient was the use of resources in achieving the project objectives? f. To assess to what extent the project objectives were achieved and what was the quality of the achievements? g. To assess the sustainability of the project interventions h. To assess the impact of the project It will require the following deliverables: 1. Inception Report outlining the evaluation methodology to be used by the evaluator and a final work plan; 2. Stakeholder workshop: The evaluator shall facilitate a learning workshop in country to present the draft report and the findings of the evaluation to the project and key stakeholders; to gather feedback on the findings and build consensus on recommendations; 3. Draft evaluation report with specific recommendations; lessons learnt and good practices; 4. Final evaluation report (max. 30 pages) incl. executive summary. Gender equality issues shall be explicitly addressed throughout the evaluation activities of the consultant and all outputs including final reports or events need to be gender mainstreamed. 3 Evaluation arrangements and professional requirements An Independent Evaluator will conduct a final independent evaluation between 30 May 2018 and 12 July The assignment is for approximately 29 work days during that period. The assignment will require travel to Sierra Leone. The country office to be visited and Action Against Hunger project staff will help in organizing accommodation, arranging meetings and transportation. Selection of the Independent Evaluator will be done by the Evaluation Learning and Accountability Unit and will be based on the strength of the qualifications provided by potential candidates through their expressions of interest for the assignment. Interested candidates should include in the expression of interest: 1. An updated CV; 2. A detailed presentation of their background and experience in Nutrition and Food Security programming, highlighting significant field experience in the evaluation of humanitarian / development projects relevant to this assignment; 3. A statement of availability for the entire duration of the assignment; 4. A daily consulting fee in EURO 5. Two work samples. 6. A proposed methodology and work plan Applicants should have a Relevant degree / equivalent experience related to the evaluation to be undertaken; Significant experience in coordination, design, implementation, monitoring and evaluation of projects; Good knowledge of West Africa context and understanding of donor requirements for Irish Aid; Ability to manage the available time and resources and to work to tight deadlines; Independence from the parties involved; 2

3 Excellent written and oral communication skills in English. Local languages are an advantage. The deadline to submit expressions of interest for the evaluation is by close of business on 20 th April Applicants should submit a complete expression of interest (including all 5 elements highlighted above) via to ELA@actionagainsthunger.org.uk and indicate: Sierra Leone SLFFH Recruitment in the s subject line. 3

4 TERMS OF REFERENCE For the Independent External Evaluation of Action Against Hunger s project Reinforcing Institutional Capacity for Treatment of Acute Malnutrition, Prevention of Malnutrition and National Sensitisation for Nutrition Security in Western Area, Sierra Leone Programme Funded by Irish Aid Summary Table Programme/Project/Mission Name Reinforcing Institutional Capacity for Treatment of Acute Malnutrition, Prevention of Malnutrition and National Sensitisation for Nutrition Security in Western Area, Sierra Leone SLE.ACTION AGAINST HUNGER Health, Nutrition, FSL, Advocacy N/A Western Area Urban (Freetown) and Western Area Rural districts 2 years Contract Number Sector Partners (if applicable) Location (country/ies, region/s) Duration Starting Date 1 June 2016 Ending Date 31 May 2018 Programme/Project/Mission English Language Donor & Contribution/s Irish Aid: 1,466,942 AFD: 389,974 Mission administering the Action Against Hunger Sierra Leone Programme/Project Responsible Action Against ACF France Hunger HQ Evaluation Type External evalution Evaluation Dates 30 May 12 July ACRONYMS 1

5 AFD CHW CSO FSG IGA IMAM IPF GoSL MAFFS MIYCF MoHS MSG OTP PWLM SUNI VSLA Agence française de développement Community Health Worker Civil Society Organization Father Support Group Income Generating Activity Integrated Management of Acute Malnutrition In-Patient Facility Government of Sierra Leone Ministry of Agriculture, Forestry and Food Security Maternal and Infant and Young Child Feeding Ministry of Health and Sanitation Mother Support Group Outpatient Therapeutic Programme Pregnant Women and Lactating Mothers Scaling Up Nutrition and Immunization Village Savings and Loan Association 2

6 1. PROJECT BACKGROUND 1.1. Map of Project Area 1.2. Rationale for the Project This project is the continuation of the project Action Against Hunger has implemented from September 2013 to October 2015 in Western Area. The areas of interventions have been chosen along with representatives of the Ministry of Health and Sanitation (MoHS), Ministry of Agriculture Forestry and Food Security (MAFFS). The specific objective of the project is to reinforce institutional capacity for quality treatment of acute malnutrition, improve preventive strategies and to raise national awareness on nutritional security in the communities. The aim of the project is to strengthen the capacities of the MoHS at local and national level to ensure quality implementation of the Integrated Management of Acute Malnutrition (IMAM) protocol following its national revision in As per phase I, this project proposal is part of a larger regional intervention in West Africa, cofinanced by Agence Française de Développement - French Development Agency (AFD), which concerns three countries for operational implementation of activities (Niger, Burkina Faso and Sierra Leone) and seven countries for advocacy actions (Mauritania, Guinea, Liberia, Ivory Coast, Nigeria, Chad, Mali). The overall goal of the project is to contribute to the improvement of maternal and child nutrition in West Africa Project Objectives Through this intervention Action Against Hunger aims to improve the nutritional status of children under-five and Pregnant Women and Lactating Mothers (PWLM) by intervening to strengthen the health system, at community and health facility level, and by implementing preventive approaches tackling the direct causes of undernutrition. Whilst this intervention represents the second phase of a project co-funded by Irish Aid, implemented in the period in Western Area, which has proven to be effective in improving the nutritional knowledge and status of the targeted communities, it also takes in 3

7 consideration the recommendations highlighted in the external evaluation conducted in September 2015, and the needs expressed by stakeholders and the communities. The specific objectives of the project are: 1. Improving the nutritional status of children under-five and Pregnant Women and Lactating Mothers (PLW) through the integration of nutrition in a strengthened health system and in preventive approaches to the direct causes of undernutrition; 2. Mobilizing contributing sectors to address the underlying causes of under nutrition in a holistic, integrated and sustainable way; 3. Enabling political, social and sectorial environment is created for the scaling-up of the fight against acute malnutrition based on the experiences of Action Against Hunger and civil society in Sierra Leone Please refer to the table below for both direct and indirect targeted beneficiaires of this project: Direct beneficiaries: 5,532 Estimated Male: 517 Estimated Female:5,015 Description 162 Health staff in PHUs Staff of 65 PHUs will undergo refresher training on the revised IMAM protocol to ensure they have the capacities to detect and manage cases of severe acute malnutrition correctly. 30 Health staff in IPFs 3 27 Staff of 2 IPFs will undergo refresher training on the revised IMAM protocol to ensure they have the capacities to detect and manage cases of severe acute malnutrition with complication correctly 200 members of Facility Management Committee and Ward Development Committee Facility Management committee members and 100 ward development committee members will be trained to help support the quality of management of acute malnutrition and to oversee the misuse of therapeutic products in their communities. 4,620 mothers from 462MSGs 0 4,620 (3,200 established mothers and 1,420 cascaded mothers) 4,620 mothers from 462 MSGs are trained and supported to conduct monthly awareness-raising at community level on maternal, infant and young child feeding practices (MIYCF) and to participate in monthly meetings at PHU level with PHU in-charge and Action Against Hunger staff 1150 mothers from 115 groups are also targeted to increase their income and improve their diet and supported to establish vegetable gardening and/or IGAs with inputs and capacity building. 200 fathers from 20 Father Support Groups 320 Community Health Workers (CHW) fathers from 20 Father Support Groups (FSGs) formed are targeted to raise awareness on nutritional and health issues in consideration of the role men play in food purchase, dietary decisions and health seeking behaviours. The same fathers will also be targeted to increase their income and improve their diet and supported to establish vegetable gardening and/or IGAs with input and capacity building CHWs are trained and supported to correctly detect and refer malnourished children from the communities to the closest PHUs where Nutrition programme exists 4

8 Indirect beneficiaries: 1,517,194 Western Area population 606, ,327 Freetown area population through improved nutritional awareness and practices, and improved access to nutritional services 1.4. Project Current Status Specific Objective 1: Improving nutritional status of children below 5 years old and Pregnant and Lactating Women through integrating nutrition in a strengthened health system and in preventive approaches to direct causes of malnutrition Result 1: Strengthen health systems to allow for better quality of care of acute malnutrition Current Status: Sphere standards achieved (Cure rates: 88.5%; Default rates: 4.3% and Death rates: 4.3%) 126 health facilities staff trained in IMAM: 90 for Outpatient Therapeutic Programme (OTP) (5 males and 85 females) and 30 for In-Patient Facility (IPF) (3 males and 27 females) 32 OTPs and 2 IPFs have functional anthropometric tools Result 2: Strengthen community mobilization to improve coverage of management of acute malnutrition Current Status: 138,826 (78.45%) children under five screened for acute malnutrition IMAM coverage (55.1%) 320 Community Health Workers (CHWs) trained in nutrition surveillance and continued to carry out surveillance activities in target communities 462 Mother Support Groups trained in Maternal and Infant and Young Child Feeding (MIYCF) MSGs held 12,353 MIYCF sessions Lead Mothers screened and referred 2,465 children under five for acute malnutrition Result 3: Specific approaches to high-impact nutrition are implemented to reduce the direct causes of under nutrition Current Status: 48 cooking demonstrations done 6 joint monitoring supervisions to health facilities done with DHMT Default rate reduced to 4.3% 20 Father support Groups (FSGs) formed Specific Objective 2: Mobilizing contributing sectors to address the underlying causes of under nutrition in a holistic, integrated and sustainable way. Current Status: Action Against Hunger has been actively participating in the technical meetings on food and nutrition security (e.g. Food Security Working Group, Nutrition Technical Working Group) and is part of the national task force conducting the Harmonized Framework (Cadre Harmonize ) analysis on semestral basis. The project supported 135 groups, 115 Mother Support Groups (MSGs) and 20 Father Support Groups (FSG), to establish Income Generating Activities (IGAs) at group level, of which 131 are operational to date. In addition, Action Against Hunger supported the establishment of 110 Village Saving and Loans Associations (VSLAs) by members from the abovementioned 135 groups and members from an additional 31 MSGs; all of the VSLAs are active to date. 5

9 Specific Objective 3: Enabling political, social and sectorial environment is created for the scalingup of the fight against acute malnutrition based on the experiences of Action Against Hunger and civil society in Sierra Leone Current Status: Action Against Hunger contributed to an increase in the number of national policy frameworks that promote nutrition sensitivity and multisectorial approach to address underlying causes of malnutrition. Action Against Hunger, together with the established SUNI Civil Society Platform and MoHS had pushed the process of the adoption of the Code of Marketing for Breastmilk Substitutes in Sierra Leone (draft regulation, MPs sensitization). Also, Action Against Hunger, together with FAO and MAFFS have provided technical support and facilitated Civil Society Organization (CSO) participation in the revision of the national agriculture investment plan to include nutrition sensitive indicators. Finally, Action Against Hunger is part of the National Working Group for the revision of the National Food and Nutrition Security Implementation Plan the main strategic document in country that is aiming to coordinate and guide multisector and multistakeholder efforts to address malnutrition. With regards to better nutrition financing, Action Against Hunger concluded budget analysis where we tracked government s expenditure and allocation for nutrition. Unfortunately, the overall budget allocation for nutrition specific interventions through the relevant line ministries slightly decreased in 2016 when compared with Lastly, Action Against Hunger has been actively engaged in facilitating civil society s role in national policy influencing processes, conducted awareness raising activities and high level events with the local civil society. 2. PURPOSE AND OBJECTIVES OF THE EVALUATION 2.1 Rational for the Evaluation This evaluation is conducted as an exercise of accountability towards the donor and the beneficiaries. It is also expected to contribute to better understand the strengths and weaknesses of the intervention, drawing lessons learnt and making operational and strategic recommendations that can be used to improve the implementation of a potential next phase or similar interventions in the future. 2.2 Objectives of the Evaluation The overall purpose of the evaluation is to assess the overall performance of the project and to determine if the intervention has reached its intended outputs and objectives. In particular, it will assess to what extent (and the reasons why) the project s outputs have contributed to the improvements in the nutritional security of children and mothers in Western Areas. 2.3 Users of the Evaluation Direct users: ACF France HQ (ACF France Pool Desk Staff), Action Against Hunger SL Country Director, Action Against Hunger Health/Nutrition Head of Department, Action Against Hunger SL Food Security and Livelihoods Head of Department, Action Against Hunger SL Advocacy Expert, Action Against Hunger SL East-Freetown Field Coordinator, Action Against Hunger Nutrition Security Project Manager and Irish Aid. Indirect users: Action Against Hunger s International Network, relevant ministries in Sierra Leone (MAFFS, MoHS, DFN), others (donors, partner organisations, federal, regional and local governments, ministries, UN agencies and Global Clusters, NGOs and NGO Consortiums as well as humanitarian learning platforms. 2.4 Use of the Evaluation 6

10 Learn from experience to develop new strategies at the global level, collect lessons learnt and good practices for future projects in country. 3. EVALUATION SCOPE 3.1. Elements covered by the evaluation The evaluation will focus on the entire project funded by Irish Aid and AFD. It will cover all geograpghical areas, looking at different levels of the intervention (community level, district level, and national level) and at the links between those levels. It will also cover all selected target groups of beneficiaries and will examine the implementation of all activities and the degree of achievement of all outputs and objectives. Finally, the evaluation should provide key recommendations to be considered for the new two year s project in the same area with a wider focus on sustainability. Moreover, it will identify and recommend potential exist strategies Elements not covered by the evaluation There are no specific elements that will not be covered by the evaluation Cross-cutting issues The evaluation is also expected to carry out a gender analysis, explore the differences in vulnerabilities between men, women, boys and girls, how the project addressed these gender equalities and how a new project can improve on this. Furthermore, the evaluation shall assess any potential negative consequenses of the project on the gender equality situation. 4. EVALUATION APPROACH AND QUESTIONS As per ACTION AGAINST HUNGER Evaluation Policy and Guidelines 1, ACTION AGAINST HUNGER adheres to the Development Assistance Committee (DAC) criteria for evaluating its programmes and projects. Specifically, ACTION AGAINST HUNGER uses the following criteria: Relevance/Appropriateness, Coherence, Coverage, Efficiency, Effectiveness, Sustainability and Likelihood of Impact 2. To the latter list ACTION AGAINST HUNGER adds an additional criterion, Design. ACTION AGAINST HUNGER also promotes a systematic analysis of the monitoring system in place within the aforementioned criteria. Evaluation questions have been developed to help the evaluator assess the project against these criteria (Refer to Annex V). The evaluator may adapt the evaluation criteria and questions, but any fundamental changes should be agreed between the ELA at ACTION AGAINST HUNGER-UK and the evaluator and reflected in the inception report. All independent external evaluations are expected to use DAC criteria in data analysis and reporting. In particular, the evaluator must complete the DAC criteria rating table (Refer to Annex V) and include it as part of the final evaluation report. 5. EVALUATION DESIGN AND METHODOLOGY This is a one-group design evaluation, examining a single intervention, without including any comparison with units (people, communities, etc.) that did not participate in the intervention. This The criterion has been rephrased to Likelihood of Impact as a thorough impact assessment is linked to the estimation of attribution, which can only be measured through experimental or quasi experimental evaluation designs. The evaluation design for carrying out a 2 performance The criterion evaluation has been would rephrased not be to suitable Likelihood to determine of Impact the as effects a thorough attributed impact to assessment the project. is linked to the estimation of attribution, which can only be measured through experimental or quasi experimental evaluation designs. The evaluation design for carrying out a performance evaluation would not be suitable to determine the effects attributed to the project. 7

11 evaluation is also expected to look at units participating in the program before and after the intervention has been in operation. This section outlines the suggested methodological approach for the evaluator to collect quantitative and qualitative data. The evaluator will to the extent possible develop data gathering instruments and methods which allow collecting sex-disaggregated data. The instruments need to make provision for the triangulation of data where possible Evaluation Briefing Prior to the evaluation taking place, the evaluator is expected to attend an evaluation technical briefing with the ELA ACTION AGAINST HUNGER-UK. Briefings by telephone must be agreed in advance Action Against Hunger HQ Briefing As part of the evaluation, the evaluator will interview HQ stakeholders to get preliminary information about the project being evaluated. Briefings by telephone must be agreed in advance Desk review The evaluator will undertake a desk review of project materials, including the project documents and proposals, progress reports, outputs of the project (such as publications, communication materials, videos, recording etc.), results of any internal planning process and relevant materials from secondary sources Inception Report At the end of the desk review period and before the field mission, the evaluator will prepare a brief inception report. The report will be written in English and will include the following sections: Key elements of the TORs to demonstrate that the evaluator will adhere to the TORs; Present the methodological approach to the evaluation (including an evaluation matrix in annex to specify how the evaluator will collect data to answer the evaluation questions) and point out the limitations to the methodology if any and the choice of sites per field visit; Provide a detailed evaluation workplan and; State adherence to Action Against Hunger Evaluation Policy and outline the evaluation report format. The inception report will be discussed and approved by the mission in Sierra Leone Field Mission Primary data collection techniques As part of the evaluation, the evaluator will interview key project stakeholders (expatriate/national project staff, local/national representatives, local authorities, humanitarian agencies, or donor representatives) as per the list in Annex IV. The evaluator will use the most suitable format for these interviews as detailed in the inception report. The evaluator is also expected to collect information directly from beneficiaries. Towards enriching triangulation, if budget and timeframe allows, the evaluator could also conduct Focus Group Discussions (beneficiaries, non-beneficiaries, key informants health workers, teachers and leaders) and household surveys. Field visits The evaluator will visit the project sites and the facilities provided to the beneficiaries (if any). Secondary data collection techniques: Desk review 8

12 The evaluator will further review complementary documents and collect project monitoring data or of any other relevant statistical data. Debriefing The evaluator shall facilitate a debriefing session in country to present preliminary findings of the evaluation to the project and key stakeholders (including National actors); to gather feedback on the findings and build consensus on recommendations; to develop action-oriented session statements on lessons learnt and proposed improvements for future Evaluation Report The evaluation report shall follow the following format and be written in English: Cover Page; Summary Table to follow template provided Table of Contents Executive Summary must be a standalone summary, describing the project, main findings of the evaluation, and conclusions and recommendations. This will be no more than 2 pages in length. Background Information Methodology describe the methodology used, provide evidence of triangulation of data and presents limitations to the methodology Findings includes overall assessment of the project against the evaluation criteria, responds to the evaluation questions, all findings are backed up by evidence, cross-cutting issues are mainstreamed and; unintended and unexpected outcomes are also discussed Conclusions Conclusions are formulated by synthesizing the main findings into statements of merit and worth, judgements are fair, impartial, and consistent with the findings Lessons Learnt and Good Practices presents lessons that can be applied elsewhere to improve project performance, outcome, or impact and; identify good practices: successful practices from those lessons which are worthy of replication; further develop on one specific good practice to be showcased in the template provided in Annex VI Recommendations Recommendations should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it both locally. They should follow logically from conclusions, lessons learnt and good practices. The report must specify who needs to take what action and when. Recommendations need to be presented by order of priority Annexes These should be listed and numbered and must include the following: Good practice template (annex VI), Evaluation Criteria Rating Table (annex V), list of documents for the desk review (annex II), list of persons interviewed (annex IV), data collection instruments and evaluation TORs. The whole report shall not be longer than 30 pages, 50 pages including annexes. The draft report should be submitted no later than 10 calendar days after departure from the field. The final report will be submitted no later than the end date of the consultancy contract. Annexes to the report will be accepted in the working language of the country and project subject to the evaluation Debriefing with ELA, at Action Against Hunger UK The evaluator should provide a debriefing to the ELA in ACTION AGAINST HUNGER-UK to discuss any issues related to the evaluation report Debriefing with Action Against Hunger HQ 9

13 The evaluator should provide a debriefing with the relevant ACTION AGAINST HUNGER HQ on her/his draft report, and on the main findings, conclusions and recommendations of the evaluation. Relevant comments should be incorporated in the final report. 6. KEY DELIVERABLES The following are the evaluation outputs the evaluator will submit to the mission in Sierra Leone : Outputs Inception Report Stakeholders workshop Draft Evaluation Report Final Evaluation Report Deadlines All outputs must be submitted in English and under Word Document format. The quality of the inception report and the evaluation report will be assessed by the mission. The evaluator is expected to follow the format, structure and length as defined under section 5.4 and 5.6 above. 7. MANAGEMENT ARRANGEMENTS AND WORKPLAN These evaluation TORs have been developed in a participatory manner, by the ELA at Action Against Hunger UK based on inputs from relevant stakeholders. The evaluator will directly report to the ELA at Action Against Hunger UK. The evaluator will submit all the evaluation outputs directly and only to the ELA at Action Against Hunger UK. The ELA at Action Against Hunger UK will do a quality check (ensure required elements are there) and decide whether the report is ready for sharing. The ELA will forward a copy to key stakeholders for comments on factual issues and for clarifications. The ELA will consolidate the comments and send these to the evaluator by date agreed between the ELA and the evaluator or as soon as the comments are received from stakeholders. The evaluator will consider all comments to finalize report and will submit it to the ELA who will then officially forward to relevant stakeholders. Once the evaluation is completed, the ELA at Action Against Hunger UK will prepare the management response follow-up form to track implementation of the recommendations outlined in the evaluation report. A review of the follow-up process will be undertaken six months after the publication of the evaluation report Tentative Workplan NOTE: Consultants are expected to work 6 days a week (either Sundays/Fridays or whatever day the field office has off will not be paid) during their consultancy contract. Travel days are not paid as they are not working days as such. Activities Evaluator Dates Day Working Days Evaluation briefing with ACTION AGAINST HUNGER /05/2018 Wednesday UK ELA Interviews with HQ /05/2018 Wednesday Desk review, preparation of field work and prepare 2 31/05/2018- Thu-Fri Inception Report 01/06/2018 Review of the Inception Report 10

14 Travel to Sierra Leone 0 (1 travel day) 02/06/2018 Saturday In country interviews with project staff 1 04/06/2018 Monday Field work, collection and analysis of secondary data & meeting with stakeholders 12 05/06/ /06/2018 Tue-Mo Debriefing Session 1 19/06/2018 Wednesday Workshop with stakeholders 1 20/06/2018 Thursday Travel back from Sierra Leone 0 (1 travel day) 21/06/2018 Friday Evaluation debriefing with ACTION AGAINST /06/2018 Monday HUNGER-UK ELA Evaluation debriefing with HQ /06/2018 Monday Draft Report 6 26/06/ /07/2018 Tue-Mo Final report on the basis of stakeholders, Mission and 3 10/07/2018- HQ comments 12/07/2018 Total: 29 Tue-Thu 7.2. Profile of the evaluator The evaluation will be carried out by an international evaluation consultant with the following profile: Knowledge in health, nutrition, food Security and Livelihood and advocacy sectors; Significant field experience in the evaluation of humanitarian / development projects; Relevant degree / equivalent experience related to the evaluation to be undertaken; Significant experience in coordination, design, implementation, monitoring and evaluation of programmes; Good communications skills and experience of workshop facilitation; Ability to write clear and useful reports (may be required to produce examples of previous work); Fluent in English Understanding of donor requirements; Ability to manage the available time and resources and to work to tight deadlines; Independence from the parties involved. 8. LEGAL AND ETHICAL MATTERS The ownership of the draft and final documentation belong to the agency and the funding donor exclusively. The document, or publication related to it, will not be shared with anybody except Action Against Hunger before the delivery by Action Against Hunger of the final document to the donor. Action Against Hunger is to be the main addressee of the evaluation and its results might impact on both operational and technical strategies. This being said, Action Against Hunger is likely to share the results of the evaluation with the following groups: Donor(s) Governmental partners Various co-ordination bodies 11

15 For independent evaluations, it is important that the consultant does not have any links to project management, or any other conflict of interest that would interfere with the independence of the evaluation Intellectual Property Rights All documentation related to the Assignment (whether or not in the course of your duties) shall remain the sole and exclusive property of the Charity. 9. ANNEXES TO THE TORs I. Project Logframe II. List of Project documents for the desk review III. Evaluation Criteria and Detailed Evaluation Questions IV. List of people to be interviewed V. Evaluation Criteria Table VI. Good practices Template 12

16 Annex I: Project Logframe Title of the Action Reinforcing institutional capacity for treatment of undernutrition acute malnutrition, prevention of malnutrition and national sensitisation for nutrition security in Western area, Sierra Leone Principal Objective To contribute to the improvement of nutritional security of children and mothers in Western Area, Sierra Leone. Intervention Logic Objectively Verifiable Indicators Sources of Verification Assumptions and Risks Specific Objective Improving the nutritional status of children U5 and PWLM (Pregnant Women and Lactating Mothers) through the integration of nutrition in a strengthened health system and in preventive approaches to the direct causes of under nutrition. Result 1.1 Strengthening the health system allows for better quality of care of acute malnutrition Reduction in the prevalence of severe acute malnutrition in the area of intervention SPHERE standards attained and progressed in all health centres supported (Cured rate > 75%, Defaulter rate <15% and death rate < 10%) 162 of health staff trained and implementing quality management of SAM 32 PHUs with functional anthropometric tools Baseline, Endline and KAP survey National SMART surveys MoHS/DHMT nutrition reports (HMIS) HMIS reports List of trained health staff Donation certificates of functional anthropometric tools to PHUs Joint monitoring and supervision report Service Level Agreement (SLA) signed as soon as the project kick-off Community Health Workers are identified and recognized by communities and health system; they are always available for training and project implementations Therapeutic food and Routine treatment for OTPs and IPFs are available at district level according to Monthly report and case load Health staff are fully following the IMAM protocol High turnover of Health staff would limit the impact of on the job coaching Result 1.2 Strengthening community mobilization improves the coverage of the management of 80% of children U5 screened quarterly in the communities of intervention IMAM program coverage is Quarterly screening reports Survey reports (i.e. SQUEAC) HMIS report DHMT members are always available to support the project implementation No outbreak of contagious 13

17 acute malnutrition within acceptable Sphere standards for urban (70%) and rural (50%) areas. 320 of community health worker (CHWs) trained and working on early detection, referral and follow up of under nutrition 462 of M2M groups trained on MIYCF messages 80% of targeted mothers demonstrate increased knowledge and practices of recommended maternal, infants and young feeding practices Result 1.3 Specific approaches to highimpact nutrition are implemented to reduce the direct causes of under nutrition Activities Activity % increase in the proportion of targeted mothers who demonstrate knowledge of timely initiation of complementary feeding 3% reduction in the defaulter rate of the IMAM program demonstration sessions conducted in 32 PHUs 8 rounds of joint monitoring and supervision done with DHMT Monitoring and supervision reports Attendance lists and training report HMIS report Joint monitoring and supervision reports Monthly ACTION AGAINST HUNGER Activity Progress Report KAP report diseases e.g. Ebola Result 1.1 Organize and conduct initial or refresher training of 162 health staff working in 65 OTPs and 30 health staff in the 2 existing IPFs for quality management of acute malnutrition Activity Organize and conduct training of DHMT staff on IMAM 14

18 Activities Result 1.2 Activity Provide technical support in the form of on the job coaching for all OTP and IPF health staff Activity Joint supportive supervision with the DHMT to supported OTPs and IPFs Activity Train and support to Facility Management Committees and Ward Development Committee members in order to contribute to the quality operation of IMAM. Activity Jointly develop with MoHS and DHMT an exit strategy Activity Support strengthening of the supply chain for RUTF through monitoring, analysis and advocacy Activity Support District Coordination Meetings Activity Provide specific materials, medical equipment and anthropometric tools through the DHMT for OTPs and IPFs Activity Development of monitoring tools for quality implementation Activity Strengthening community awareness through community meetings Activity Organize and conduct refresher training of 320 CHWs in the operational areas Activity Support monthly meetings of CHWs; and provide tools to CHWs for detection and referral (Tally sheet, referral slip, etc.) and for sensitisation and awareness. Activity Organize and conduct 1 Refresher training of 320 MSG leaders 15

19 Activities Result 1.3 Activity Organize and conduct training for 142 cascaded MSGs Activity Support monthly meetings for 142 cascaded MSG groups vision of tools and provision of materials (T-shirts, bags, id tag, etc.) Activity Support monthly meetings for 320 MSG groups and provision of tools and provision of materials (T-shirts, bags, id tag, etc.) Activity Pilot Behaviour Change and Communication (BCC) approaches to the established 12 community health club; 20 teenage mothers, and 20 fathers support groups Activity Conduct KAP survey at baseline and end line Activity Conduct a baseline SQUEAC analysis Activity Provide support to DHMT (logistic and small items support) Activity Conduct quarterly joint supervision with MOHS (national level) Activity Organize quarterly cooking and food demonstrations in 32 PHUs targeting pregnant women during Ante-Natal Care days or mothers of children under 2 for complementary feeding Activity Dedicate technical support to Directorate of Food and Nutrition (MOHS) to support on specific technical aspects (i.e. OTP technical issues and Nutrition surveillance) Activity Establish a partnership with the national NGO, Food and Nutrition Security Initiative Sierra Leone (FANSI-SL) Activity Conduct a barrier analysis and develop Behaviour Change Communication sensitisation materials to promote key nutrition behaviours based 16

20 on findings of the barrier analysis Specific Objective 2 Mobilizing contributing sectors to address the underlying causes of under nutrition in a holistic, integrated and sustainable way Result 2.1 Improve the comprehension of vulnerabilities and the consideration of nutrition in contexts analysis, in surveillance and in Early Warning Systems in order to inform an integrated response to under nutrition multi sectoral causes. Result 2.2 Food security and livelihood complementary activities with high impact on nutritional state are set for under five children and PWLM Determinants of nutritional vulnerability are well understood and taken into account at national level analysis. 75% of Harmonized Framework analysis integrate phasing of nutrition situation when valid data are available. Early Warning System bulletins integrate analysis on nutrition 55% of targeted beneficiaries increase their FCS 75% of targeted beneficiaries increase their monthly income National Early Warning System Bulletins; Sierra Leone Harmonized Framework Analysis (Sheet 3). Early Warning System and Harmonized Framework analysis continue to be performed in country. National Early Warning System Bulletins; Sierra Leone Harmonized Framework Analysis (Sheet 3). Valuable data on nutrition are collected and available for analysis. Baseline/End-line surveys No major food crisis. No major economic crisis and/or market disruption. No major hazards (drought, floods, epidemic). Targeted beneficiaries have access to land and water for vegetable production, otherwise ACTION AGAINST HUNGER will support other IGAs rather than farming. No bye-law restrictions on gathering of people, movement of goods and people, and 17

21 markets closure are put in place by the Government. Result 2.3 Good practices in contributing sectors are adopted for sustainable behaviour change to improve the nutritional status at Community level. 80% of caregivers have knowledge and adoption of at least 3 key IYCF messages 80% attendance of targeted mothers at IYCF sensitisation and counselling sessions KAP Joint monitoring report HMIS Mother support groups functional No outbreak of contagious diseases e.g. Ebola Availability of IYCF materials Implementation of the no touch policy Activities Activity Result 2.1 ACTION AGAINST HUNGER is present in technical meetings on nutrition and food security and provides technical support particularly to the Early Warning System and Harmonized Framework analysis. Activities Activity Result 2.2 Support existing 70 MSGs with training and inputs to enhance the vegetable production also for income (marketing) and/or to set up other income generating activities Activity Support additional 45 new MSGs with training and inputs to enhance vegetable production (for both HH consumption and income) and/or to set up income generating activities. Activity Support 20 Father Support Groups with training and inputs to set up Income Generating Activities Activity Establish 110 new VSLA groups and provide them with equipment and training Activities Activity Result 2.3 Develop counselling cards for MSGs using live photos 18

22 SO 3 Enabling political, social and sectorial environment is created for the scaling up of the fight against acute malnutrition based on the experiences of Action Against Hunger and civil society in Sierra Leone Result 3.1 Nutrition is better integrated in strengthened health systems in Sierra Leone Result 3.2 Nutrition is better integrated in contributing sectors policies in Sierra Leone Result 3.3 The coordination of the civil society is strengthened so that they can effectively influence government and other stakeholders for better commitments and improved Increase of in the number of political frameworks in the country which integrate a nutrition multi sectorial approach. Public documents and health policies/strategies reflect nutrition integration in the health systems At least one newly drafted or revised political framework better integrates nutrition The overall budget allocation for nutrition specific interventions through the relevant line ministries increases by 0.5% by 2018 At least 2 advocacy actions are organised in collaboration with the national civil society actors in nutrition Government documents (i.e. policies, protocols, directives, budgets) Ministry of Health documents (i.e. policies, protocol, directives, training agenda, essential drugs list) National sectorial policies, nutrition multi sectorial plans Advocacy and policy documents and activities reports from Action Against Hunger Mission and other national civil society Government continues to be supportive on scaling up nutrition in the country No major crisis occur leading government to diverge attention and funds from nutrition SUN Secretariat and SUN CSP continue to play a role in coordinate efforts (advocacy and programmes) to scale up nutrition 19

23 Activities Result 3.1 Activities Result 3.2 Activities Result 3.3 accountability organisations Activity Develop and disseminate case studies, briefings and other evidence based documents on Action Against Hunger nutrition specific programmes and on the improvement of nutrition through a better integration of nutrition interventions within the health system (i.e. role of CHWs, MUAC self-referral by mothers, RUTF supply chain monitoring). Activity Develop communication material for the mission and media campaigns to raise awareness on nutrition Activity Support to MoHS in communication and advocacy activities, particularly during public health campaigns Activity Develop and disseminate case studies and briefings on Action Against Hunger nutrition sensitive programmes and their contribution to scale up nutrition Activity Conduct analysis on the nutrition sensitivity of national policies and programmes in contributing sectors and dissemination of the results Activity Conduct analysis on nutrition sensitive and nutrition specific expenditures of SL government, and dissemination of report Activity Active participation in various coordination meetings and working groups at national and district level to advocate for inclusion of nutrition into different sectorial plans Activity Develop communication activities and materials to raise awareness on the multi-sectorial nature of nutrition (i.e. jingles and radio discussions etc.) Activity Active participation in the SUNI Taskforce Meetings at national level (Secretariat and Civil Society Platform) Activity Support to SUNI Civil Society Platform advocacy activities, particularly in influencing national policies and in occasion of national and international events (i.e. World Breastfeeding Week, Maternal and Child Health Week, International Food Day, etc.) 20

24 Activity Support to other civil society organisations that are active in nutrition in their advocacy activities and campaigns; 21

25 Annex II: List of documents for the desk review The following documents will be reviewed by the evaluator during the desk review phase: Document Action Against Hunger s Evaluation Policy and Guideline Project Proposal and Amendments Sierra Leone National Nutrition Survey 2014 GoSL Basic Package of Essential Health Care Services 2015 Population and Housing 2015 Essential Medicines List ACF WASH Nutrition manual ACF SL Country Strategy Anemia Strategy 2017 NHSS Strategis Plan SL National Strategy Ebola Recovery SL CHW Policy SL Micronutriient Survey 2013 SL Anemia Strategy 2017 SL SARA 2017 FMC strategy 2017 Action Against Hunger: Aligning Health and Nutrition Strategy Project Interim Reports Project Baseline survey reports Project Barrier Analysis report 2017 Project Semi Quantitative Evaluation of Access and Coverage (SQUEAC) report 2017 Mother to mother support groups: a guidance document, 2016 Sierra Leone Livelihood Zoning Report 2016 Comprehensive Food Security and Vulnerability Analysis (CFSVA) 2015 Sierra Leone National Nutrition Survey report 2017 Irish Aid Project Final External Evaluation 2015 ACF Gender Policy and Toolkit AAH Agro-ecology position paper 2017 AAH Our Expertise Food Security and Livelihoods Tracking Government Expenditure on Nutrition in Sierra Leone 2017 Support, Protect and Promote Breastfeeding: Parliamentarians Workshop (Final Report) Civil Society Statement: To integrate nutrition in Sierra Leone s ICADEP Final draft of Inclusive Comprehensive Agriculture Development Plan (ICADEP) Food and Nutrition Security Implementation Plan Nutrition Sensitivity Analysis of ICADEP Policy and Plan Overview Sierra Leone (Done by REACH) Nutrition Security in Sierra Leone Case Studies (1 st year and 2 nd year) Mama Salone Newsletters (Volumes 7 to 10) 22

26 Annex III: Evaluation Criteria and Detailed Questions To assess the project against each evaluation criteria, the evaluator will respond to the following evaluation questions: Design: A measure of whether the design is logical, allows for RBM and include a sustainability strategy involving local partners and beneficiaries Are beneficiaries needs (by sex and age) well identified and in which way? What was the level of beneficiary participation in project design? Is gender properly taken into account in project design? Are project objectives and indicators SMART? Are sources of verification realistic? Is the design of the exit strategy realistic? Is there a good design of the M&E system in place? Were the recommendations from the external evaluation of Phase 1 of the project Reinforcing institutional capacity for treatment of acute malnutrition, prevention of malnutrition and national sensitisation for nutrition security in Freetown Peninsula, Western Area taken into account? Were the recommendations from the barrier analysis taken into account? Relevance/Appropriateness: A measure of whether interventions are in line with local needs and priorities (as well as donor policies, thus increasing ownership, accountability, and cost-effectiveness) Were the actions undertaken relevant and appropriate given the local context and needs of the target population? Was the assistance relevant and appropriate in relation to the practices / culture of the target population? To what extent were the needs of beneficiaries and stakeholders take in to account in project implementation? Coherence: A measure of whether interventions are consistent with existing interventions, global and national policies and strategies to ensure consistency, maximize synergies and minimize duplication Are other stakeholders informed or aware about ACTION AGAINST HUNGER activities/approach/strategy of the project? How activities of this project have been integrated with other ACTION AGAINST HUNGER sectors/ programs in the operational area? The project had three technical areas, nutrition, food security and advocacy and communication, to what extent were there synergized and integrated during implementation? Coverage: A measure of whether interventions meet the need to reach major population groups facing life threatening suffering wherever they are Were the most affected groups covered with the limitation of the resources available? Was the geographical coverage of the project appropriate? Were beneficiaries correctly and fairly identified and targeted and to what extent were local communities involved in beneficiary identification? How the targeting was understood or perceived by local communities? Were gender and vulnerable populations with in the target community considered in ACTION AGAINST HUNGER s assessment/identification of the beneficiary and in the implementation of the project? Did the project include special components for women, if so; were these systematically designed and monitored during implementation? Efficiency: A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results, includes value for money analysis 23