The Syria Co-ordinated Accountability and Lesson Learning (CALL) Initiative Terms of Reference for the Thematic Synthesis of Evaluative Reports Background The Syria crisis has entered its fifth year with no end in sight and with numbers of displaced people not seen since World War II. Despite the magnitude, severity and impact of this crisis, no system-wide evaluative effort has been trigged to assess the performance of the humanitarian system thus far. To compensate for the absence of a system-wide evaluative exercise, the inter-agency Syria Coordinated Accountability and Lessons Learned (CALL) Initiative 1 was launched in October 2013 to contribute to improved evidenced-based decision making through a number of activities aiming at promoting learning and accountability. The initiative was originally established by the Steering Group for Inter-Agency Humanitarian Evaluations (IAHEs) with the following key deliverables: the Syria Learning Portal 2, the Common Context Analysis, a mapping exercise of evaluative studies 3 on the Syria crisis followed by a gap analysis twinned to a synthesis report using a Common Evaluation Framework (CEF). The Syria CALL initiative was based on the assumption that given the geographic spread and scale of the crisis humanitarian actors would produce a large number of evaluations to assess their programmes. In this scenario the Syria CALL initiative would have played a coordination, communication and advocacy role to support system learning and accountability. This assumption proved overoptimistic, as evaluative efforts have been limited and uneven in coverage, and the finalisation of a rigorous CEF unpractical. Nevertheless, a thematic synthesis and gap analysis of evaluative studies so far mapped, five years into the conflict, will provide a useful overview of the areas of learning and accountability efforts undertaken thus far. This will also give an indication of the evolution of the humanitarian response, its challenges and achievements. It will also be useful in identifying areas of further inquiry. 1 The Syria CALL was established in mid-2013 by the IASC Steering Group for Inter-Agency Humanitarian Evaluations (IAHE) and is managed by an inter-agency Management Group composed of WFP, UNHCR, UNICEF, OCHA and ALNAP. The Syria CALL initiative s key objectives are to a) enhance the quality and use of evaluative activities; and b) increase learning and knowledge sharing for improved humanitarian response. Other key outputs of the initiative include the Syria Crisis Common Context Analysis (Update forthcoming covering June 2014 and August 2015), and the ALNAP s Syria Portal (http://www.syrialearning.org). 2 The portal features a library containing: evaluative materials as well as different types of evaluative and analysis products. These are: After action & learning reviews, Articles, Audio-visual material, Blogs, Books, Conference, training & meeting documents, Evaluation reports, Factsheets and summaries, Lessons papers, Plans, policy and strategy, Presentations, Programme/project reviews, Research, reports and studies, Tools, guidelines and methodologies. 3 Definition of evaluative studies: Evaluative research seeks to assess or judge in some way, providing useful information about something other than might be gleaned in mere observation or investigation of relationships. From: http://changingminds.org/explanations/research/design/evaluative_research.htm. Evaluative studies can range from informal appraisals to full-fledged evaluations.
Introduction OCHA 4 seeks an experienced consultant to undertake the last output of the Syria CALL initiative, namely the gap analysis and thematic synthesis of evaluative studies. The thematic synthesis will be based only on evaluative studies in response to the Syria crisis from 2012 to present. A mapping exercise has been completed in preparation for the analysis. The mapping includes documents such as needs assessments, vulnerability assessments, studies on the impact of the crisis, and other studies. The assignment will be undertaken between October 2015 and January 2016 for a total of approximately 50 working days. Objectives of the thematic synthesis report and gap analysis The overall purpose of the Syria CALL initiative is to contribute to improved evidenced-based decision making, and improved humanitarian intervention in Syria and the region and/or other large complex emergencies. The specific objectives of the gap analysis and thematic synthesis are to: provide an overview of recurrent findings, issues, themes and gaps emerging from the analysis and identify areas where further inquiry is required; provide an evidence-based analysis and lesson learned to operational decision making, current and future response policy and programming; and, contribute to the debate on humanitarian action in conflict affected settings in the context of the forthcoming World Humanitarian Summit 5 (2016). To achieve these objectives the synthesis will attempt to answer the following key questions: What are the recurring areas / themes covered in the finding and recommendations produced in evaluation reports (comparable in scope / topic covered)? Are there areas, themes, cross-cutting issues that emerge as being consistently undercovered / more neglected in the evaluative materials included in the synthesis? Documents under Analysis Over the last year, the Syria CALL Management Group 6 has been retrieving, collecting and organizing documents concerning a wide-range of project and programme initiatives implemented by humanitarian actors responding to the needs of affected Syrians and host communities in Turkey, Jordan, Lebanon and Iraq. For the purpose of this assignment twenty (20) evaluative reports will be synthesised, based on the mapping s classification. Of the 20 documents classified as evaluative reports, some are broad scope evaluations (they look at the performance of an agency or donor in a number of 4 OCHA seeks the services of a consultant on behalf of the inter-agency Management Group composed of UNICEF, WFP and ALNAP. 5 Key debates at the WHS surround its four core themes: Humanitarian Effectiveness, Transformation through Innovation, Serving the Needs of People in Conflict, and Reducing Vulnerability and Managing Risk 6 The Syria CALL Management Group is composed of UNICEF, WFP, ALNAP and OCHA.
key areas), while others are narrowly focused (for example, they evaluate the effects of specific interventions such as health or cash interventions in a specific area or population). Table 1: Evaluative Reports under Analysis Year Agency Title 1 2015 OCHA Evaluation of OCHA s Response to the Syria Crisis (forthcoming) 2 2015 UNICEF Evaluation of UNICEF s Humanitarian Response in Syria and the Sub-Region (forthcoming) 3 2015 WFP An Evaluation of WFP s Regional Response to the Syrian Crisis, 2011-2014 4 2015 UNHCR Independent Programme Evaluation of the UNHCR response to the refugee influx in Lebanon and Jordan 5 2015 NRC Evaluation of the Norwegian Refugee Council's Lebanon Host Community Shelter Programmes 6 2015 OXFAM Humanitarian Quality Assurance: Jordan 7 2015 DANIDA Evaluation of the Strategy for Danish Humanitarian Action 2010-2015, Case Study Syria 8 2015 DANIDA The Syrian displacement crisis and a Regional Development and Protection Programme: Mapping and meta-analysis of existing studies of costs, impacts and protection, 2014. 9 2015 ACTED/ Failing Syria: Assessing the impact of UN Security Council REACH Resolutions in Protecting and Assisting Civilian in Syria 10 2015 OCHA Operational Peer Review 11 2014 IFRC Real-Time Evaluation of the Response to the Syria Crisis 2012-2014 12 2014 OXFAM Humanitarian Quality Assurance: Lebanon. Evaluation of Syria Crisis Response 13 2014 HelpAge Evaluation of HelpAge International s Programme Intl 14 2014 AusAid Evaluation of Australia s Response to the Syria Crisis 15 2014 IRC An Impact Evaluation of the 2013-2014 Winter Cash Assistance Program for Syrian Refugees in Lebanon 16 2014 ICVA Review of NGOs Experience with the Syria-Related Pooled Funds 17 2013 UNHCR A real-time evaluation of UNHCR s response to the Syrian refugee emergency 18 2013 ACAPS Syria Needs Analysis Project (SNAP): External mid-term review 19 2013 Hayata Destek Evaluation Study: Support to Life s Response to Syrian Refugee Crisis, Turkey 2012-2013 20 2013 DEC DEC Syria Crisis Appeal. Response Review: Final Report Audience The audience for this study is multi-faceted given the objectives of the Syria CALL initiative. The primary audience is an internal one, meaning members of the IASC Steering Group for Inter-Agency Humanitarian Evaluations 7. The secondary audience includes external policy 7 Members: UNICEF, UNHCR, WFP, FAO, World Vision, WHO, OCHA, IFRC, etc
makers, other parts of the UN, international and national NGOs, think tanks, research institutions and others. Expected Deliverables 1. Inception Note and Analytical Framework An inception note of no more than 10 pages will provide an understanding of how the consultant intends to undertake the study. The consultant will propose the most appropriate options for the analytical framework considering the SHARP (2012-2014), Syria HRP (2015), the 3PR and the Syria Crisis Common Context Analysis (2014). The analytical framework will be developed using an inductive research approach, namely the recurrent themes will emerge from the data. 2. Gap Analysis and Thematic Synthesis Report The gap analysis will identify thematic gaps against the broad analytical framework. The analysis of the different sectors covered, scope, geographic variation, and others, will contribute to a better understanding of what has been done on the ground and identify important areas of gaps for further enquiry. The thematic synthesis report will summarise recurring trends, findings, themes, conclusions and recommendations from the 20 evaluative studies. It will also provide an overview of the type, scope and nature of the documents under analysis. The report comprising the both the gap analysis and the thematic synthesis should not exceed 40 pages without annexes. Annexes will include: Terms of Reference. Documents reviewed: the bibliography should have a brief description of the document and a separate comment on how useful the document was for the analysis. Mapping of quantitative incidence across the documents universe of each themes looked into. Management arrangements The consultant will be contracted by, and report to, OCHA on behalf of the CALL Management Group. The Management Group review and approve all outputs of this assignment. Timeline Timing Sept 2015 Sept 2015 Oct 2015 Activities ToRs finalised Advertisement for consultancy Consultant selected and contracted Consultant submits inception note
Timing Early Dec. 2015 Dec /Jan 2015 Activities First draft Review of draft report by Management Group Revision of draft and submission of final report Competency Requirements The assignment will require the services of a consultant with the following experience and skills: Understanding of the Syria crisis and its impact on neighbouring countries and the whole region; Experience undertaking evaluations in humanitarian settings and excellent understanding of evaluation principles and norms; Experience undertaking synthesis of large numbers of documents, using quantitative and qualitative research approaches; Proven experience writing clear, neutral analysis of complex and sensitive issues; Broad understanding of the UN system and humanitarian issues; Excellent English writing skills. How to apply Interested and qualified candidates are requested to apply for this consultancy in inspira, the online recruitment system of the United Nations Secretariat, accessible through this link: https://inspira.un.org. The inspira Job Opening (JO) number for this consultancy is 48366, title of the JO: Syria CALL: Synthesis Report. Deadline is 02 October 2015. The application must also include a maximum two pages proposal on how the consultant envisages undertaking the assignment in order to be considered for the job. Please indicate in your cover letter your availability and daily rate to undertake the Terms of Reference above. Applications submitted without a daily rate and a proposal will not be considered. Only short listed candidates will be contacted.