Terms of Reference FINAL PROJECT EVALUATION Strengthening humanitarian action in urban areas by promoting settlement approaches and effective engagement with local stakeholders Executive Summary Donor: European Commission - Directorate General - Humanitarian Aid and Civil Protection - ECHO Project Title: Strengthening humanitarian action in urban areas by promoting settlement approaches and effective engagement with local stakeholders Project Duration: 24 months (1 September, 2016 31 August, 2018) Project Locations: Global with field pilots in Central African Republic, Haiti, Iraq, Lebanon, Syria and Uganda. Main Project Objective: Specific Project Objectives: Expected Results of the Project Objective of the Evaluation: To contribute to more effective, efficient, and inclusive humanitarian response in urban settings To provide a replicable and evidence-based capacity to optimize the use of area and settlement-based approaches for humanitarian planning and response to urban crises. 1. Settlement-based and multi-sector response is applied in at least 5 urban crises 2. Synergies with and capacity of key local responders, notably municipal service providers, are enhanced in at least 5 urban crises 3. Guidance tools and training on adopting settlement-approaches for Global humanitarian platforms (at cluster and/or inter-cluster level) are developed, through lessons learnt and best practices from the case studies. The main objective of this final evaluation is to provide the project partners and ECHO with an external assessment of the results against the predefined indicators, as well as contribute to building the evidence base of the applicability of the settlements/area approach. Specifically, this will entail a review of the status, relevance and performance of the project as compared to the project proposal document, and to assess the results and impacts, with overall guidance questions of: Has the adoption of settlement-based and multi-sector approaches in the project pilots resulted in improved efficiency and effectiveness of humanitarian action? Have synergies with- and capacity of- key local responders, notably municipal service providers, been enhanced in the project pilots? Have advocacy efforts, guidance, tools and training on adopting settlementapproaches for humanitarian actors been adopted and applied as a result of the project? The evaluation will research the outputs and impacts of the project at operational and policy levels. Operational outputs and impacts will be informed through review of pilots, whilst policy outputs and impacts will be informed through review of advocacy and convening roles, primarily undertaken at the global level.
Methodology Period of field assessment: Expected completion date: Evaluation results are envisaged to inform future programming through reviewing lessons and best practices identified through the project s timeframe, whilst evaluating their validity and identifying any further lessons from the project implementation. This will be achieved through: measuring changes in the set indicators; analysing the experiences gained, (both technically and managerially); and recommending the approaches and methodologies for further dissemination and programming. Key informants discussions, focus group discussions, survey, direct observations, review of project documents, including all documents developed as part of pilots. Approximately one month, with the exact start date to be collectively agreed with project partners The evaluation will be conducted during Summer 2018, with the latest completion date of mid-september 2018. Exact dates will be collectively agreed with project partners Background to the Project By 2050, over 70% of the global population will live in urban areas. This urbanisation trend is accompanied by an increasing prevalence and intensity of natural and man-made disasters in urban centres. Despite this increase, the humanitarian system has struggled to understand and effectively engage with the complex socio-economic dynamics and governance structures existing within cities. Humanitarian response planning and coordination primarily occur at the sectoral or cluster level, missing an important holistic understanding of a complex urban environment. Sub-national actors, including municipal authorities, service providers and civil society, often remain excluded from humanitarian planning and coordination processes. Acknowledging these challenges, with the goal of strengthening humanitarian response in cities affected by crisis, IMPACT, ACTED, United Cities and Local Governments (UCLG) and Cites Unies France (CUF) have partnered in the framework of this project to implement a 2-year pilot of area/settlement-based approaches, while launching a new inter-agency initiative between IMPACT and ACTED, AGORA. The project specifically promotes the use of settlement approaches - also known as area-based or neighbourhood approaches (defined as applying a geographically targeted, multi-sectoral, and participatory approach) to inform better humanitarian planning and response in urban areas, and the launch of a new initiative AGORA to promote such approach beyond the project duration. Targeting five cities affected by crisis 1, the project works with humanitarian coordination platforms and with local responders (notably the municipalities and basic service providers) to promote a multi-sectoral response organised by settlement, which is mainstreamed within the humanitarian coordination system and supports local response capacity. Whilst maintaining a degree of flexibility to ensure relevancy to the specific context, the approach, at an operational level, follows a series of steps: 1. Delineating & identifying crisis affected areas within a city 2. Undertaking comprehensive area based assessments 3. Develop area-level response plans 4. Establish or support area-level coordination mechanisms 5. Strengthen capacity of local actors through peer to peer municipal support 1 The target pilot cities include Bangui, Central African Republic; Jeremie, Haiti; Mosul, Iraq; Kampala, Uganda; Saida, Lebanon; and Ar-Raqqa, Syria.
Parallel to operationalising the approach in identified cities in crisis, the project includes a dedicated focus of advocating for and promoting the application of settlement approaches in global fora and processes. Within the framework of the project: ACTED has contracted directly to IMPACT Initiatives and UCLG/CUF, and remains responsible for the oversight of IMPACT and UCLG/CUF activities. For the purpose of this external evaluation, IMPACT will designate a staff to act as a focal point for the consultant. IMPACT acts as the leading partner in terms of program strategy and implementation, including the selection and successful implementation of country pilots and global advocacy and awareness raising activities. CUF represents UCLG and provides technical support to municipalities affected by crisis through sourcing expertise from other municipalities. Project Results and targets Principal Objective: The project will contribute to more effective, efficient, and inclusive humanitarian response in urban settings Specific Objective: To provide a replicable and evidence-based capacity to optimize the use of area and settlement-based approaches for humanitarian planning and response to urban crises. Number of case studies with evidence of good practice (4) Number of global coordination platforms that endorse and commit to applying papers, tools and Mechanisms (2) % of respondents reporting the use of settlement-approach effectively - developed or facilitated by the project - improves the capacity of aid actors coordinate with local response stakeholders (70%) Result 1: Settlement-based and multi-sector response is applied in at least 5 urban crises: Humanitarian coordination mechanisms support the delineation of settlements and assessment of basic services in 5 urban crises; Settlement-based multi-sectoral response plans are piloted in 5 urban crises Result 2: Synergies with and capacity of key local responders, notably municipal service providers, are enhanced in at least 5 urban crises: Number of urban crises part of the pilots in which local response capacity is assessed. Response plans in the 5 pilots propose concrete ways to support local responders. Result 3: Guidance tools and training on adopting settlement-approaches for Global humanitarian platforms (at cluster and/or inter-cluster level) are developed, through lessons learnt and best practices from the case studies: 5 case studies are peer reviewed 3 outputs are developed (trainings, modules, guidelines) Key Project Stakeholders In order to ensure the success of the project, IMPACT has involved many stakeholders in the implementation of the project, as follows:
Result 1: Across the 5 identified cities in crisis, a diverse range of operational actors, as listed below, were involved in areas based assessment, planning and coordination activities. United Nations operational and coordination agencies: OCHA, UNHCR, IOM, UNICEF, WFP, FAO, UNDP, UN-Habitat; Humanitarian Coordination mechanisms operating at the country-level: Humanitarian Country Teams, Inter-Cluster Coordination Group, Shelter Cluster, CCCM Cluster, Food Security Cluster, WASH Cluster; International organisations: Such as the IFRC; International NGOs: For example NRC, IRC, Mercy Corps, DRC, SCI, etc; (Targeted NGOs vary depending on the field pilot) National authorities: City and municipal Mayors within the targeted cities as well as relevant line Ministries/local governance entities; National stakeholders: Such as civil society responding to the crisis and the private sector; Affected populations in targeted countries: Both displaced population and host communities. Result 2: Relationships were developed with city and municipal authorities in target cities, in addition to municipal associations and networks offering technical assistance National authorities, including the Mayors of the targeted cities as well as relevant line Ministries/local governance entities; Municipal representatives (international): Including the United Cities and Local Government (UCLG) Task force on Local and Regional Government Disaster Response, the Swedish Association of Local Authorities and Regions (SKL), International Cooperation Agency of the Association of Netherlands Municipalities (VNG) Result 3: Relationships were established with global humanitarian actors championing improved humanitarian response in urban contexts. Global Humanitarian Coordination Mechanisms: Including the Inter Agency Standing Committee Reference group on Meeting Humanitarian Challenges in Urban Areas (IASC MCHUA), the Inter Cluster Coordination Group and Global clusters (Shelter Cluster, CCCM Cluster, Food Security Cluster, WASH Cluster) Networks and initiatives: Including the Global Alliance for Urban Crises Municipal representatives (international): Including the United Cities and Local Government (UCLG) Task force on Local and Regional Government Disaster Response Key Components of the Final Evaluation The main objective of this final evaluation is to provide the project partners and DG-ECHO with an assessment of the results against the predefined indicators, as well as contribute to building the evidence base of the applicability of the settlements/area approach. This will entail a review of the status, relevance and performance of the project as compared to the project proposal document, and to assess the results and impacts, with overall guidance questions of: Has the adoption of settlement-based and multi-sector approaches in the project pilots resulted in improved efficiency and effectiveness of humanitarian action? Have synergies with- and capacity of- key local responders, notably municipal service providers, been enhanced in the project pilots? Have advocacy efforts, guidance, tools and training on adopting settlement-approaches for humanitarian actors been adopted and applied as a result of the project? The evaluation shall follow the following framework:
1. Relevance: The appropriateness of project objectives to the problems that it was supposed to address, and to the physical and policy environment within which it operated. It should include an assessment of the quality of project preparation and design i.e. the logic and completeness of the project planning process, and the internal logic and coherence of the project design. Was the selection of pilot cities appropriate for applying the area based approach? Were the project interventions (methodologies, activities and outputs at both operational and policy level) relevant to achieve the project objectives? Did the project adequately adjust to changing situations or contexts? 2. Efficiency: The fact that the project results have been achieved at reasonable cost, i.e. how well inputs/means have been converted into activities, in terms of quality, quantity and time, and the quality of the results achieved. This requires comparing alternative approaches to achieving the same results, to see whether the most efficient process has been adopted. Was the project managed efficiently, in terms of human capital, staff, financial and other resources versus the results? Were project partners implementation structures appropriate? Were external partners and affected populations sufficiently involved in the project implementation? Was there feedback from project implementers to project partners? What were the external constraints to achieving better efficiency and how well were they mitigated? Were synergies capitalised on with other actors (local and international) involved in similar projects? What evidence is there (if any) that learning (M&E), from this project or previous projects, were incorporated into the project s implementation strategies? How has the relationship been between the project partners and local government authorities? 3. Effectiveness: An assessment of the contribution made by results to achievement of the project purpose, and how assumptions have affected project achievements. This should include specific assessment of the benefits accruing to target groups. To what extent did the project achieve its specific objective To provide a replicable and evidence-based capacity to optimize the use of area and settlement-based approaches for humanitarian planning and response to urban crises.? Were the expected results realised: - Settlement-based and multi-sector response is applied in at least 5 urban crises - Synergies with and capacity of key local responders, notably municipal service providers, are enhanced in at least 5 urban crises - Guidance tools and training on adopting settlement-approaches for Global humanitarian platforms (at cluster and/or inter-cluster level) are developed, through lessons learnt and best practices from the case studies. 4. Impact: The effect of the project on its wider environment, and its contribution to the wider policy or sector objectives (as summarized in the project s overall objective).
Did the project ultimately contribute to the overall objective stated at the outset? i.e. To contribute to more effective, efficient, and inclusive humanitarian response in urban settings What were, if any, the innovative and successful approaches that are of relevance to other actors active in humanitarian response in urban crises? What, if any, were the unintended impacts of the project intervention, both positive and negative? What were the external constraints to achieving better effectiveness and how well were they mitigated? Are the apparent impacts attributable to the project s interventions? 5. Sustainability: An assessment of the likelihood of benefits produced by the project to continue to flow after external funding has ended. To what extent are the area based approach mechanisms implemented in the pilot cities able to continue following the project s cessation? To what extent were the risk mitigation and sustainability plans in the project proposal, in addition to unforeseen risks, addressed during project implementation? How realistic were these? What evidence is there to suggest the project s interventions and/or results will be sustained after the project end? What are the possibilities for replication and extension of the project s outcomes? In addition to the above, the project evaluation will consider the performance of the project in relation to specific commitments, to be mutually agreed between the consultant and project partners, within the Core Humanitarian Standard (CHS) on Quality & Accountability. Methodology of the Final Evaluation The evaluation will be based on the findings and factual statements identified from review of relevant documents and discussions with key interlocutors. It is anticipated the consultant will physically travel to two of the five pilot cities, collectively agreed with the project partners. The remaining three pilot cities will be reviewed via distance. Document review: Documents to be reviewed include, but not limited to: Original project proposal Key internal documentation (communications, memos, etc) Interim donor report Internal progress reports Assessment and response plan reports for the targeted cities in crisis Technical peer-to-peer reports Discussions with key interlocutors: The consultant is expected to facilitate discussions with key partners in targeted cities and at the global level. Key interlocutors include representatives from: The Inter Cluster Coordination Group and Global clusters (OCHA, Shelter Cluster, CCCM Cluster, Food Security Cluster); The Global Alliance for Urban Crises members;
United Cities and Local Government (UCLG) representatives; National stakeholders such as authorities (City and municipal Mayors (or their delegates) within the targeted cities), civil society, etc. in targeted pilot cities; Humanitarian Coordination mechanisms operating at the country-level, such as Shelter Cluster, CCCM Cluster, etc.; International organisations such as the IFRC, UN-OCHA, UNHCR, etc. International NGOs, including IRC, JIPS, IIED, etc; Field mission: In order to successfully undertake the above, the consultant, in agreement with the project partners, will be required to undertake one to two field missions to targeted pilot cities. Detailed Outputs of the Assignment/Deliverables The evaluator shall provide IMPACT with the following deliverables: 1. Inception Report outlining: - Review design and methodology (review tools, data collection, organization and data analysis etc.) - A detailed plan activity 2. Final report in the English language including the following elements. Executive Summary Methodology Findings Conclusions, Lessons Learned, Best Practices, and Recommendations Annexes It should be tightly drafted, and usable as a free-standing document. It should be short, not more than five pages. It should focus on the main analytical points; indicate the main conclusions, lessons learned and specific recommendations. Cross-references should be made to the corresponding page or paragraph numbers in the main text that follows. The methodology section should detail the tools used in the evaluation, the locations, the sampling, limitations faced, the dates, the team composition, and other pertinent facts. The findings section should present the results of the evaluation in an objective and non-judgmental way that gives an honest portrayal of the project as a whole. Included in the findings should be a discussion of how well the project achieved each of the five DAC criteria: relevance, effectiveness, efficiency, impact, and sustainability. These should be presented as a separate final chapter. Both positive and negative conclusions should be made, as relevant. Wherever possible, for each key conclusion there should be a corresponding recommendation. Recommendations should be as realistic, operational and pragmatic as possible. Terms of Reference of the evaluation Names of the evaluators and their companies Logical Framework matrices (original and improved/updated) Map of pilot cities/locations, if relevant List of organizations consulted Literature and documentation consulted Other technical annexes (e.g. statistical analyses) The primary target audience of this evaluation includes the project donor (DG-ECHO) and project partners IMPACT, ACTED, United Cities and Local Governments (UCLG) and Cites Unies France (CUF).
The secondary target audience includes key interlocutors and partners at both global and country pilot levels. Please note that both inception and final reports are subject to the project partners approval before considered as final deliverables. All comments from project partners should be addressed before the report is considered completed. Duration This assignment is planned to be accomplished within approximately 20 working days exact start date to be collectively agreed with project partners. The following schedule and deliverables are suggested: Deliverables Review of program activities, implementation policies and reporting mechanisms Deliver and present Inception note Field visits and collection of program data for analysis Delivery of Draft Assessment Report for incorporation of project partners input within 7 days. Delivery of Final Assessment Report, integrating project partners inputs Schedule 2 Days 1 Day 12 Days 3.5 days 1.5 Days Throughout the course of the evaluation, frequent meetings with project partners will be required. Expertise Requirements The consultant should have the following background: Post graduate qualifications in development studies, humanitarian action, international relations or relevant area; Experience in project Monitoring and Evaluation, in particular related to urban crises response and coordination; Experience in implementing and/or coordinating humanitarian response in an urban context; Strong knowledge of humanitarian coordination mechanisms; Strong analytical skills and ability to clearly synthesize and present findings; Good written and oral English essential; Knowledge of French an advantage. Expression of Interest Interested candidates should submit their offer to tenders@impact-initatives.org no later than 12 am, 15/06/2018. The offer shall include the following: Letter of Expression of Interest (no longer than 2 pages) CV (no longer than 2 pages each) Planning proposal including suggested methodology and timeframe Financial offer Applications will be assessed according to the abovementioned four documents in the application. Shortlisted applicants will then be required to undertake an interview process (skype/teleconference).
Any missing document will lead to the direct disqualification of the applicant. Delayed applications will be automatically rejected. Additional details Insurance coverage: The consultant must be fully and independently insured (both personal and professional liability), evidence of an appropriate insurance certificate is to be submitted to the project partners. Taxation: Please note that the consultancy firm will have to comply with all government rules and will be responsible for government taxes. Field mission arrangements: Field mission details will be mutually arranged with the consultant and project partners. Field mission costs (i.e. international travel, in-country accommodation etc) will be covered by project partners. Whilst undertaking field missions the consultant will follow project partners safety and security procedures and will be accommodated in project partners guest house.