Evaluation of the India Earthquake response

Transcription

1

2 International Federation of the Red Cross Red Crescent Societies Evaluation of the India Earthquake response Inception Report FINAL September 2001 Simon Lawry-White Page 2

3 CONTENTS EXECUTIVE SUMMARY EXECUTIVE SUMMARY... I 1. INTRODUCTION CLIENT TERMINOLOGY CONTEXT OF THE INCEPTION REPORT AIMS OF THE INCEPTION REPORT METHODOLOGY KEY SUCCESS FACTORS IN THE FEDERATION S RESPONSE CONTEXT OF THE INDIA EARTHQUAKE RESPONSE FEDERATION/PARTICIPATING NATIONAL SOCIETIES INDIA RED CROSS SOCIETY INDIAN AUTHORITIES EXTERNAL FACTORS BRIEF LITERATURE REVIEW APPEAL SITUATION REPORTS END OF MISSION OTHER EVALUATIVE EXERCISES OTHER DOCUMENTS DOCUMENT COLLECTION PREPARING THE TERMS OF REFERENCE FOR THE MAIN EVALUATION LEARNING POTENTIAL IMPACT VERSUS MANAGEMENT PROCESS PROCESS... ERROR! BOOKMARK NOT DEFINED TOPIC STRUCTURE PRIORITISATION...9 EVALUATION OF THE INTERNATIONAL FEDERATION OF THE RED CROSS RED CRESCENT SOCIETIES INDIA EARTHQUAKE RESPONSE, DRAFT TERMS OF REFERENCE... I CONTEXT...I PURPOSE...I OBJECTIVES...II Transition to Rehabilitation...ii Capacity Building of the India Red Cross Society...ii Federation Management Systems...ii Disaster Response Mechanisms... iii METHODOLOGY...III Literature Review / Data analysis... iii Beneficiary consultation/survey... iii Peer review...iv Interviews and Correspondence...iv Workshops...iv COVERAGE... IV EVALUATION TEAM... IV EVALUATION REPORT...V Summary Report: A stand alone document summarising key findings and recommendations... v Main Report: Limited to 30 sides, not including the summary report and appendices.... v TIME SCALE...V Page i

4 5. CONDUCT OF THE EVALUATION BENEFICIARY SURVEY EVALUATION TEAM DURATION AND COST PUBLICATION CONCLUSION...11 APPENDICES 1 Literature Review and Inception Report Terms of Reference 2 Possible topics for evaluation 3 Lists of Interviewees 4 Schedule of documentation 5 Donor national societies questionnaire analysis (partial) 6 Some key dates/events 7 Related evaluative exercises 8 International Red Cross Red Crescent Disaster Response and Rehabilitation policies (extracts), plus Code of Conduct for the International Red Cross Red Crescent/NGOs in Disaster Response Programmes Page ii

5 EXECUTIVE SUMMARY The report was commissioned by the Federation Secretariat s Monitoring and Evaluation Department on the basis of Terms of Reference jointly agreed between the Head of the Asia Pacific Department and the Director of Disaster Management and Coordination. The Secretariat decided in May 2001 that the evaluation should be divided into two stages, a formative evaluation, from which this report arises and a main evaluation to be conducted later. The two stage process was adopted because of a perceived need to create a clear focus for the evaluation, without which the potentially broad scope of the evaluation might have made it unmanageable. The findings in the inception report are provisional and will be tested by the main evaluation. The main output of the inception report is a Terms of Reference for the main evaluation. The TOR prioritises those areas where lesson learning will have the greatest impact on the future performance of the Federation in emergency response and rehabilitation. These include: The impact and quality of the Federation s assistance to beneficiaries Transition to Rehabilitation Capacity building of the India Red Cross Federation Management Systems (specific aspects) Disaster Response Instruments (specific aspects) There are many learning points from those areas not included in the TOR that are already captured in this report and which can be actioned internally by the Secretariat without the need for input from external consultants. The earthquake of magnitude 7.9 that struck Gujarat on January was the most severe to hit India in more than fifty years. Some 20,000 people lost their lives and hundreds of thousands of homes and public buildings were destroyed and damaged. On January 30, the Federation Secretariat launched an Appeal for CHF 25.6 million, to which donors responded generously, with some CHF 40.4 million in goods, services and cash pledged and donated. The India Red Cross (IRCS) decided within two days of the earthquake to invite the Federation of Red Cross and Red Crescent Societies to mount a disaster response in Gujarat in partnership with the IRCS. In total, some fifty National Societies made donations and sent delegates to assist with the operation. The emergency operation was successful in many respects. The operation met its targets in terms of relief distributions and may well have exceeded the target of 300,000 beneficiaries. The Federation responded rapidly to the emergency situation, with the first personnel reaching the epicentre at Bhuj within 48 hours. Emergency Response Units (ERUs) and their equivalents were mobilised rapidly by eight National Societies to provide medical, water and sanitation, logistics, relief, and telecommunications services. All are reported to have performed well. Lessons have already been learned from the experience in Gujarat for future deployment of ERUs and the Field Assessment and Co-ordination Team. A 310-bed Red Cross ERU hospital was operational at Bhuj less than a week after the earthquake. The hospital became the flagship of the Federation operation and an important part of a public relations success for both the IRCS and the Federation as a whole. The Federation was the first agency to reach Bhuj with a satellite phone and was able to provide first hand information to the international media before any other agency. The Federation continued to provide information for the national and international media for several weeks in several languages. The period for the emergency operation was set at 120 days. During this time 228,000 blankets, 117,000 tarpaulins, 35,000 tents, 53,000 kitchen sets and 61,000 water containers were distributed. 160 tonnes of highenergy biscuits were despatched but were largely unused and are now in storage in India. Only minimal amounts of unsolicited and inappropriate relief goods were received. Page i

6 The largely successful emergency operation was made possible by the coming together of several factors, which are explored in the report. The personal commitment of staff and delegates was crucial, together with the high levels of coordination between National Societies, both in the field and at headquarters. The Federation lost momentum in its move to the rehabilitation phase. The Federation started its rehabilitation planning within two weeks of the disaster and an apparently successful assessment mission was followed by a Partnership Meeting in Delhi during which donors made soft pledges of more than CHF 50 million towards rehabilitation. Since then, rehabilitation planning has proceeded slowly. The rehabilitation appeal was finally launched on July 9 and has produced an uncertain response from donors. There is important learning to be had from these events, which seem to arise from shortcomings in the way the transition to rehabilitation was conceptualised, planned and resourced and a lack of effective coordination models within the Federation. The only aspect of rehabilitation to make progress during the transition phase was the dams repair programme, which had rehabilitated more than 90 surface dams by the end of the emergency period. This largely undocumented project is said to have been greatly appreciated by local communities and, with the good monsoon rains this year, will potentially have benefited many thousands of people. The report recommends that the evaluation should include a beneficiary survey to gauge how effective and appropriate the communities affected felt the Red Cross Red Crescent relief, medical and water/sanitation assistance to have been. This is a new type of exercise for the Federation. A feature of both the emergency and transition phases has been a lack of human resource at key points, both in Geneva, Delhi and Bhuj. The TOR for the evaluation includes a re-examination of the funding rules for additional temporary resources for emergency response. The TOR also includes a review of the respective roles of the newly established Disaster Management and Coordination and Programme Coordination Division, as there is currently some misunderstanding about whether DMC runs the emergency operation on behalf of the geographical department for some time, or whether the geographic department runs the operations from the outset with the DMC in support. The report raises questions about the development of the capacity the India Red Cross Society for disaster response as a result of its participation in the Gujarat emergency response and in the future rehabilitation programme. This is addressed in the evaluation TOR. Page ii

7 1. Introduction 1.1. Client This inception report has been produced for the International Federation of the Red Cross Red Crescent Societies Geneva Secretariat by an external consultant under the direction of the Secretariat s Monitoring and Evaluation Department. The Terms of Reference were agreed jointly between the Head of the Asia/Pacific Department and the Director, Disaster Management and Coordination Division. The consultant s main contact points for the conduct of the consultancy were the Programme Officer for the India Earthquake, Asia/Pacific Desk and the Monitoring and Evaluation Officer in the Monitoring and Evaluation Department Terminology The Federation is a complex organisation and even those working within it are sometimes confused about terminology. For clarity, the following nomenclature has been adopted for this report: Name IRCS The Federation Federation Secretariat Meaning adopted The India Red Cross Society, including its national HQ and state branches All Red Cross and Red Crescent Societies, including IRCS, plus the Secretariat and its regional and other offices The Geneva office of the Federation plus, in this case, the South Asia Regional Delegation (SARD), the India Operations Centre (IOC), its operations and Federation contracted delegates in Bhuj, Ahmedabad and Delhi 1.3. Context of the Inception Report From May 2001, the Secretariat was prompted by staff and by at least one National Society to undertake an evaluation of the Federation response to the Gujarat earthquake as soon as possible so as to capture the experience of the delegates and staff directly involved. In practice, most of the delegates engaged in the operation left some time ago. It is now more important to plan the evaluation properly than to rush into the full evaluation without due consideration of its focus and objectives. Many of the former delegates can be reached by phone or in due course and memories will not be significantly dulled by a further few weeks delay. Most of the key personnel in the IRCS, the Geneva Secretariat and National Societies will still be available for interview. The Secretariat decided in June 2001 that there should be a two-stage approach to the evaluation, given the scale of the operation, and uncertainty about the objectives of an evaluation and how to proceed with it. It was recognised that the exercise might become unmanageable if all stakeholders requirements were included and so it was decided to commission a literature review and an inception report to provide a clearer focus for the evaluation. The two-stage approach has proved a difficult idea to sell to the IRCS and Federation personnel in India. Some delegates cannot see the point of doing it twice and the India Red Cross seemed unimpressed by the process. Nevertheless, it is hoped that the analysis in this report will help to provide the desired strategic focus to make the best use of the resources to be set aside for the evaluation Aims of the Inception Report The Terms of Reference for this consultancy are included as Appendix 1. In summary, the purpose of this assignment in relation to the Federation s response to the Gujarat earthquake was to: Summarise the literature available Set the TOR for the main evaluation Page 1

8 Provide options for how to conduct the evaluation The TOR has evolved as the assignment has progressed, so that the inception report in its current form puts more emphasis on the strategic considerations of how to add value using the evaluation and much less on the literature review. This change in emphasis was agreed with the Asia/Pacific Desk and the Monitoring and Evaluation Division in July, after the first week of the assignment. The period for review in the inception report is from 26 January to 31 May This covers the 120-day emergency period and half of the transition to rehabilitation period Methodology This consultancy was undertaken between July and September 2001 and has proceeded as follows: Stakeholder interviews, one week in Geneva and a further week in Delhi/Ahmedabad (Bhuj was not visited) Collection of documentation in Geneva and Delhi and from the Internet and other sources Feedback sessions with Federation personnel in Delhi and Geneva, to improve the consultant s analysis Questionnaire to donor national societies on the performance of the Federation Secretariat, the IRCS and their own National Society in the India earthquake operation Discussions of the draft report with principal stakeholders and reshaping of the report to the final product The inception report has been compiled in a total of four weeks by a single consultant. All findings are tentative and are based on a synthesis of interview feedback and from a review of documentation. As far as possible, balanced judgements have been applied to the information collected, but there are bound to be some shortcomings and gaps in the analysis from this short exercise. Firm conclusions must await the full evaluation. The draft report presented a number of topics for possible inclusion in the Terms of Reference (TOR) for the main evaluation. The draft was discussed with the Directors of Disaster Management and Coordination and Programme Coordination, as well as with the Asia Pacific Desk and the Evaluation and Monitoring Department. The proposed Terms of Reference included in this final version of the report take account of their feedback and updates, and corrections have been made in the light of comments from a number of interviewees. 2. Key success factors in the Federation s response The Federation response to the Gujarat earthquake was amongst its largest operations since the Goma crisis in All parties agree that the response to the emergency was excellent in many respects. Several factors combined to make a fast, major and effective response possible. The IRCS, the Secretariat and the National Societies all played a part, as did the Indian Government, and several external factors outside the Federation s control. These factors have been summarised below, and besides a recognition of the positive contribution made by many actors, this section also shows just how many aspects needed to come together to make the response successful from the Federation s viewpoint. The absence of one or more of the inputs described below would have hampered the operation Context of the India Earthquake response India is subject to frequent disasters, especially flood and drought. The Kutch District of Gujarat State, the epicentre of the January 26 earthquake, is in the highest risk area in India for earthquakes 1. At magnitude 7.9, this earthquake was the worst in India for 50 years. In the recent past India has been reluctant to ask for international assistance. After the earthquake the Government did not appeal for assistance but welcomed offers of help. Over the past decade the Indian Red Cross (IRCS) and the Federation have not worked closely together, though this situation has been improving in recent years. 1 The Kutch region lies in Zone V on the Seismic Zoning Map of India prepared by the Indian Standards Institution Page 2

9 Until recently the management capacity of the IRCS has been declining, however some state and district branches remain relatively strong in their disaster response capabilities Federation/Participating National Societies The objective of the main Appeal was to Provide an initial target beneficiary caseload of 300,000 earthquake victims (60,000 families) with essential shelter and cold weather items, health and medical services, safe water and other urgently needed relief supplies. By the time the main Appeal was launched, more than 20 National Societies and four government agencies had already made contributions or pledges. Ultimately the Appeal was more than 140% funded, partly because of highly effective fund raising by National Societies. The emergency operation fully achieved its objective in terms of funds raised, materials supplied and probably in the numbers of people assisted. Informants were unanimous that the Emergency Response Units (ERUs) deployed to the main operation in and around Bhuj functioned well and in a coordinated fashion. All but one ERU was up and running within six days of the disaster. Since 1994, the Federation has been accumulating experience in the effective deployment of ERUs. The Spanish Basic Health Care ERU and the British Logistics ERU were new for this operation and both performed well according to Secretariat sources. The Federation provided competent coordination of the ERUs and other bilateral efforts according to feedback from National Societies including the IRCS. National Society ERUs worked together effectively. Norwegian and Finnish teams combined to provide a 310 bed referral hospital (not 350, as commonly reported), while the German WatSan ERU provided water to the hospital. The French Red Cross installed a water treatment plant in the Japanese Red Cross clinic at Sukhpur. One of the achievements in Federation coordination was that all Red Cross interventions, whether or not they were deployed bilaterally, became part of the coordinated Federation operation. All parties agree that the South Asia Regional Delegation (SARD) reacted swiftly and effectively to the emergency and put experienced personnel into the situation from the start. Following a meeting in Delhi on the day of the quake, two members of SARD and one senior member of the IRCS went to Ahmedabad. Two of the three then went directly on to Bhuj, while the third remained in Ahmedabad to provide coordination and logistical support, alongside the Gujarat State IRCS Branch. The Head of Regional Delegation (HoRD) took over as interim team leader in Bhuj on the departure of the FACT leader on February 16, and stayed on to brief the new Team Leader who arrived on February 20. The second team leader carried on the field management of the operation effectively for the next three months. The Regional Delegation also proved highly adaptable, with the Regional Disaster Preparedness Delegate switching from assistant to the FACT leader to acting HoRD while the HoRD was in Bhuj. (Some National Societies found the rationale for this swapping of personnel hard to follow). The earthquake struck on a Friday and National Societies were in discussion with the Secretariat over the weekend. The weekend period allowed the Secretariat to draw up plans with the IRCS and other National Societies and delayed the media onslaught on the Secretariat until the Monday by which time most parts of the Movement were already moving towards a coordinated response. The Federation s role in coordination proved the more valuable because it was harder for the National Societies to operate India in unaided by the Federation and the IRCS. The readiness of National Societies to coordinate and be coordinated may also indicate that lessons were learnt from the more fragmented Federation response to the Turkey earthquakes. In addition to the logistics coordination supplied by the Federation Secretariat, National Societies coordinated their own logistics closely between themselves and to an unprecedented level, according to the Secretariat. Page 3

10 Good relations already established between the Secretary General of IRCS and the new SARD HoRD were key in the Federation being able to facilitate the links between the IRCS and other National Societies, according to both the IRCS and SARD. correspondence and mission reports indicate that the Field Support Team (FACT 2 by another name) leader was well respected. The FACT seems to have been highly effective in starting up the operation under difficult circumstances. The ERUs had experienced team members, many of whom had worked together before, and most knew the FACT leader. It was reported that they trusted each other and knew what they could expect of one another. The team spirit created in the first weeks was carried forward even when the first teams left and less experienced people came. Most of the delegate End of Mission reports indicate a high level of job satisfaction, while at the same time revealing high work loads, stress and illness. Given the rigors experienced by the delegates and staff, it is perhaps remarkable that there was only one early return and one medivac amongst the total of 250 delegates (estimated). Given the difficult working circumstances that delegates and IRCS were operating under, a high level of personal commitment on the part of the delegates must have been one of the keys to success. Individuals reportedly adapted to the situation, undertaking whatever tasks were required, sometimes bending Federation procedure to get things done. In the Geneva Secretariat, experienced personnel provided the necessary leadership, including the Director of DMC and the two Asia/Pacific Senior Desk Officers. In Bhuj, almost all the Federation operations were on one large site at Lalan College. This facilitated the high levels of cooperation that have been absent in some previous operations where National Societies were physically dispersed. Following the collapse of the government hospital in Bhuj during the earthquake, the Gujarat State Government requested a 1000 bed hospital. This gave the Federation an opening for the referral hospital deployment, which became the flagship of the operation and one of the keys to the Federation public relations success. Once established, the hospital comprised 13 wards, 310 beds, an operating theatre, an OPD tent, a pharmacy and a medical warehouse. Without high levels of cooperation between National Societies including the IRCS, the referral hospital in Bhuj could not have functioned as well as it appears to have done. The Federation undertook large-scale relief distributions, though those interviewees most closely involved consider it took too long to reach the target for daily distributions. Almost all donated materials were distributed, with the exception of high-energy biscuits, which it turned out, were not required. Distributions were made over long distances to outlying villages. There was very limited pilfering of relief goods. The support by the telecomm ERUs provided by the Austrian and Spanish Red Cross proved effective, with mobile phone and VHF links especially valuable. In logistics, the British Red Cross ERU also proved effective, though it reached Bhuj 2-3 days after the optimal time. In the shipment of goods, Bhuj, Delhi, Geneva and National Societies pulled together to coordinate flights. The supply of unsolicited and inappropriate goods was limited, less than 5% of the total - better than some previous operations, though still leaving room for improvement. The Red Cross was one of the first agencies to establish an operation in Bhuj and the only one with proper international communications in the first few days. 3 The Federation had a media presence from the outset, greatly aided by being the first international agency to site with a satellite phone. This allowed the regional information delegate to become the prime source for international media, giving over a hundred interviews in the first week. International media started to move their satellite dishes away after a week but the 2 Field Assessment and Coordination Team, one of the Federations disaster response instruments. To avoid any appearance that the FACT might be taking the coordination role from any national organisation, it was decided to label the FACT as the Field Support Team. The generic term FACT is used in this document. 3 As perspective on Federation claims to be first : 1. The India Government s own response dwarfed the achievements of the Federation both in speed and scale 2. The affected area was huge and other agencies were active in other parts unseen by the Federation 3. Other agencies were on the scene, for example, It is reported that VHAI (supported by Christian Aid) was active in Bhuj from 28 January and built up quickly to an impressive total of 11 (medical) teams by 30 January (from the DEC Monitoring Report, 23 March 2001) Page 4

11 Federation managed to keep the operation s profile high in the international and national media by deploying seven information delegates, coordinated by the Federation, to give interviews in several languages over the next month and more. Federation information delegates also helped to provide a very high profile for the IRCS. A decision was made to use some of the surplus funds from the earthquake appeal to fund the rehabilitation of dams during the so-called transition phase. This has been deemed highly successful and has kept the reputation of the Red Cross high while the slow rehabilitation planning has put the good standing of the Federation at risk. 90 dams were completed by 31 May and the total by the end of August exceeded 150 (no precise figures are to hand), potentially benefiting hundreds of thousands of people. With the arrival of good monsoons after a four year drought, these dams are now full, making their contribution to improving water supplies all the more evident. The Secretariat decided to suspend the use of some of its normal management procedures in order to allow the rapid response to proceed. Normal financial and personnel approval procedures were set aside for at least the first month of the operation, without which the rapid large scale response would probably not have been possible. The process of subsequently catching up with the requisite paperwork is said by all relevant interviewees to be largely complete, but the consultant has not tested this. The supply of personnel to the India earthquake operations was in part possible by diverting Federation delegates from other programmes in India and beyond. This was good for the operation but not necessarily for the sending programmes. (The impact on other programmes has not been reviewed here) India Red Cross Society The IRCS Secretary General took a brave decision in inviting the large-scale participation of the Federation and other National Societies, which went against the trend of the previous years. Even so, there was still some apprehension that the Federation would take the limelight and the IRCS would be left in the shadow of its foreign partners. In the event, the Federation seems to have been successful in highlighting IRCS achievements in the national and international media. The Deputy Secretary (Medical) acted as coordinator and IRCS counterpart for all Federation functions in Bhuj for the whole period of the operation. Many interviewees commented on the vital contribution she made to the integration of the Federation and IRCS. She came into the operation after only five days in the Indian Red Cross, having just been recruited for an American Red Cross HIV programme. The Honorary Organising Secretary of the Gujarat State Branch provided effective facilitation in Ahmedabad and continues to give his time as a volunteer. The Gujarat State branch in Ahmedabad played a key role in facilitating the operation by making the right connections with the Government and other authorities. The Red Cross hospital was seen as so successful in the earthquake operation that Gujarat State authorities now want the IRCS to have an ERU type hospital (of around 60 beds) for immediate deployment within the State in times of disaster. The IRCS has direct links with the government, with the President of India as the President of the IRCS and the (national) Minister of Health as Chairman, providing an inside track and favoured status with the government but also the potential constraints of government rules on, for example, employment. The contribution of the IRCS may have been underestimated by the Federation because of a weak district branch, with a lack of counterpart staff from the National HQ or Gujarat Branch and limited reporting of IRCS activities outside the main operations centre at Bhuj. The IRCS contribution to the Federation operation included the provision of local volunteers and volunteers from other states, about 40 at any one time at the height of the relief operation and perhaps 300 in total. Volunteers fell into two categories those deployed from other states in India and already part of the IRCS and those recruited and paid locally and not formerly part of the Red Cross but still termed volunteers. More than one delegate commented that the volunteers were hardworking and motivated. The IRCS also provided a link between the Federation operation and the local authorities. Page 5

12 In addition to the Federation operation, IRCS state and district branches made their own contributions and interventions, including: - Bhavnagar branch set up an emergency 100-bed hospital in Bacchau and community kitchen for 5000 people and operated them for two weeks - Volunteers from several states in locations in Gujarat other than Bhuj - 30,000 IRCS blankets and 150 tents were flown into Bhuj by the Indian Air Force - Substantial in-kind contributions from within India via the IRCS Ahmedabad warehouse, (partial list of financial and in kind donations supplied is included in the files under Logistics) - IRCS medical teams from Mumbai, Pune and Madya Pradesh were active in Ghandidham and Anjar - About $350,000 was collected at national HQ, with about 50% of the balance remaining to be spent as of early July, according to the Deputy Secretary General units of blood per day were collected and distributed in the earthquake area - The IRCS set up a tracing service with support from the ICRC and the American and British Red Cross The Kutch District Branch was initially helpful but became obstructive, seemingly when the limelight and control moved away from them. Some interviewees felt that local branch antagonism was an on-going hindrance. The consultant s impression is that the overall contribution of the IRCS may have been underestimated in Federation reporting while its role in the Bhuj operation was overestimated for PR reasons. The whole operation was done in the name of the Indian Red Cross and the IRCS gained very strong recognition nationally from the operation, partly due to the Federation s efforts to promote them Indian authorities The Indian Government mounted a massive response in life-saving and medical treatment, including primary treatment of over 100,000 people within two days of the disaster and before the Red Cross Hospital was established. Over 200,000 troops were deployed to the earthquake zone. The actions of the authorities also made the Federation response possible, including: - State government s provision of free fuel and trucked in water, without which the ERU hospital and water units would not have been able to function - Indian army and air force cooperation was essential - Bhuj air strip was available near the epicentre and proved adequate for the many large international transports - India and Pakistan cooperated in allowing flights over Pakistan - The customs authorities were helpful, with minimal interference and documentation required, and visas waived in the early phase - Permission for Federation VHF radio frequencies was granted by the police immediately on request - Fibre optic and GSM networks were restored within 48 hours, though the mobile network had only 24 lines and only SMS messages could be used in the early days - Electricity was restored to Bhuj within three days of the earthquake 2.5. External Factors Gujarat State is prosperous by Indian standards and funds have continued to flow in from the Gujarat business diaspora. Local people were said to be generally resilient, honest and proud, sometimes refusing relief goods they did not require. Depopulation reduced the burden on services. Survivors fled the earthquake area to stay with relatives in Mumbai or elsewhere, mostly because of uncertainty over the safety of the buildings still standing, and because of the aftershocks (100 over magnitude 5) that produced an on-going sense of unease. They were expected to return but had not done so by the end of the emergency phase. Page 6

13 The security risk to personnel and property was relatively low compared with other Federation emergencies. Standard Chartered Bank acted quickly to allow the Federation to make cash withdrawals in Ahmedabad, though the Federation has no status agreement in India. 3. Brief Literature Review In line with an agreed change to the TOR, a full literature review has not been attempted. Some comments are provided here on documentation Appeal The Appeal process comprises three documents, the Alert, Preliminary Appeal and main Appeal, on which all subsequent reporting is based. These documents have provided an adequate basis for National Societies to pledge and then contribute well in excess of the sums requested. The Appeal document defines its objective as the provision of basic services. There do not appear to be any further planning documents for the operations, nor are there any other documents specifying how success is to be measured, other than the provision of relief needs assessed by numbers assisted. Some of the ERU deployment orders further define the services to be provided in those specific cases. Is success measured therefore just in inputs (supplies, money) and outputs (achieving the beneficiary caseload)? Delivery outputs are set for a number of recipients (300,000) but there are no qualitative criteria described in the Appeal, which is the only document setting out the objectives of the operation. The Appeal document alone does not, in the consultant s view, provide an adequate basis for subsequent evaluation. Documents setting out Federation policy and other codes and standards should provide a basis for qualitative evaluation but these were hardly mentioned in any interviews or in the operations documentation reviewed so far (see Appendix 8 Federation Policy and Codes of Conduct) Situation Reports Situation Reports (sitreps) provide the most thorough record of events, have been written initially on a daily basis, then weekly, then biweekly. They also include up to date financial reports. They are written largely for donors and therefore tend not to be self-critical. The sitrep is also directly linked to the Appeal, and so naturally focuses on the objectives of the Appeal rather than other Federation activities. They do not appear to have captured a full picture of all Red Cross Red Crescent activities in Gujarat, especially as far as IRCS activities are concerned (see 2.2 above). In addition to the sitreps, there were frequent progress reports from the team leaders in Bhuj, which provided raw material for the sitreps and which appear to give a more open appraisal of successes and failures. It is not clear whether these were part of any formal monitoring system End of Mission End of mission reports provide a potentially rich source of data for the main evaluation. Only 15 EOM reports were located in this study but Field Personnel report that they are holding many more. There must be other reports with the sending National Societies that the evaluation could usefully collect and analyse. Some of the reports offer direct and valuable feedback on the Federation operation while others appear to offer purely personal reflections. There is a standard format for these reports but some delegates did not follow it. Bilateral delegates are under no obligation to provide copies of their reports to the Federation and organisational learning is probably being lost as a result. Page 7

14 3.4. Other Evaluative Exercises In reviewing relevant documentation, other completed or planned evaluations have been considered to see what learning can be gleaned from elsewhere. A schedule of other evaluations and reviews is included as Appendix 7. Some of them are available in the document collection Other documents Financial and logistics tracking reports for this operation seem to be both comprehensive and timely 4. Press releases and news stories have been assembled. The IFRC Internet site provides comprehensive information on all aspects of Federation disaster response, policy with regard to emergency response and rehabilitation, and detail of ERUs Document Collection Four categorised lever-arch files of documents collected during this consultancy have been passed over to the Asia/Pacific Department. They will provide a starting point for the main evaluation team s literature search. The documents are listed in Appendix Preparing the Terms of Reference for the Main Evaluation 4.1. Learning Potential An evaluation of the Federation s India earthquake operation has the potential for significant learning and improvement, because: The many national societies involved all stand to gain from the lessons learnt in a full evaluation Now that there is a more open relationship with the India Red Cross, there are lessons to be learnt from the way that the IRCS, the Federation and individual National Societies worked together Against the backdrop of the apparent success of the emergency phase, minds are more likely to be open to absorbing lessons about organisational shortcomings For these reasons, investment in a full evaluation can be justified as long as it is properly focused, planned and managed, and the Federation is committed to making use of the findings to improve its practice and knowledge base Impact versus Management Process In deciding the scope of the evaluation, the Federation will need to balance its interest in management processes against a review of the impact of the Federation interventions on beneficiaries in Gujarat. The TOR for this inception phase emphasised the management capacities of the IRCS and the management processes and systems in the Federation Secretariat, while the terms beneficiary and impact do not appear. In general there is a greater emphasis on process than impact in the TOR and this is reflected in discussions in Geneva and feedback from the main donor National Societies. Interviews revealed a surprising lack of interest in the impacts of the operations on beneficiaries. This may reflect the absence of impact objectives for the emergency response within the Appeal. It may also be that the Federation s attention is diverted away from beneficiary-based assessment because of the effort and systems development required to create a timely and properly coordinated response amongst the Federation s constituent elements. Against this trend, it is recommended that the Federation rethinks its seeming lack of commitment to beneficiaries perspectives on the services being provided in their name. This is not to lessen the importance of organisational or management issues but there is a case for being less internally focused. The draft TOR below aims to provide a balance between external and internal perspectives. 4 The timeliness of financial reports and the availability of financial data on the IFRC Intranet/Internet seem to have improved significantly since the consultant s last IFRC consultancy assignment in Page 8

15 4.3. From Draft to Final report In the draft report, possible topics were grouped into two sections, one for higher priority topics and the other for lower priority, on the basis on the consultant s analysis of interviews and documentation. Comments were then invited from those who had been interviewed and the draft report was discussed with key informants in the Geneva Secretariat, including Directors of Disaster Management and Coordination and Programme Coordination, Director and Desk Officers of the Asia Pacific Department and the Monitoring and Evaluation Department. This final report includes a proposed Terms of Reference designed to include only the highest priority topics, while the rest of the analysis from the draft report now forms Appendix 2 of the final inception report. (An exception is the section on Federation Management Systems in the TOR, where the contents are drawn from a selection of topic headings in the Appendix). The topics in the draft TOR take into account stakeholder feedback on the draft report but the selection and format are based on the consultant s judgement. To achieve sufficient focus, the TOR excludes some items that were recommended in the draft report. It is recommended that these items are taken up for review internally by the Secretariat outside the scope of the external evaluation. The TOR tries to achieve a balance between width (covering the strategic topics) and depth (investigating with enough rigour to create a quality result without over-stretching the organisation s ability to manage the exercise). In finalising the TOR in advance of the main evaluation, the Secretariat may decide to add or subtract topics, using Appendix 2 as required. The final scope will also depend on the budget available. Some indicative costs are given in the section on Conduct of the Evaluation below Topic structure Each of the topics in Appendix 2 is structured to include: discussion of why this topic is relevant strategic importance of the topic for the Federation selected questions that might be addressed by an evaluation skills the evaluators will require 4.5. Prioritisation Topics in Appendix 2 are prioritised in descending order of importance, again based on feedback and an analysis of the stakeholder interviews, literature review, National Society questionnaires and the consultant s best estimate of where the greatest learning might come from. Topics are not prioritised according to their importance to the earthquake operations but rather their strategic significance for learning and improvement. The Secretariat s current level of expertise in any one area of evaluation is not considered as part of the ranking. It is assumed that the Secretariat will either equip itself or commission those with the right expertise to cover the scope of the evaluation. Page 9

16 Evaluation of the International Federation of the Red Cross Red Crescent Societies India Earthquake response Context Draft Terms of Reference The magnitude 7.9 earthquake that struck Gujarat on January was the most severe to hit India in more than fifty years. Some 20,000 people lost their lives and hundreds of thousands of homes and public buildings were destroyed and damaged. The Federation launched an Appeal for CHF 25.6 million, to which donors responded generously, with some CHF 40.4 million in goods, services and cash donated. The India Red Cross (IRCS) decided to invite the Federation of Red Cross and Red Crescent Societies to mount a disaster response in Gujarat in partnership with the IRCS. In total, some fifty National Societies made donations and sent delegates to assist with the operation. The emergency operation is considered to have been successful in many respects. The operation met its targets in terms of relief distributions and may well have exceeded the target of 300,000 beneficiaries. The Federation responded rapidly to the emergency situation, with the first personnel reaching the epicentre at Bhuj within 48 hours. Emergency Response Units (ERUs) and their equivalents were mobilised rapidly by eight National Societies to provide medical, water and sanitation, logistics, relief, and telecommunications services and support. A 310-bed Red Cross ERU hospital was operational at Bhuj less than a week after the earthquake. The Federation was first to site with a satellite phone and was able to provide first hand information to the international media before any other agency. The period for the emergency operation was set at 120 days. During this time 228,000 blankets, 117,000 tarpaulins, 35,000 tents, 53,000 kitchen sets and 61,000 water containers were distributed. 160 tonnes of highenergy biscuits were despatched but were largely unused and are now in storage. Only minimal amounts of unsolicited and inappropriate relief goods were received. The Federation lost momentum in its move to the rehabilitation phase apparently due to shortcomings in the way the transition to rehabilitation was conceptualised, planned and resourced and a lack of effective coordination models within the Federation. An assessment mission in February was followed by a Partnership Meeting in Delhi during which donors made soft pledges of more than CHF 50 million towards rehabilitation. Since then, rehabilitation planning has proceeded slowly, with the Federation planning poorly coordinated. During the transition phase, a dams repair programme had rehabilitated more than 90 surface dams by the end of the emergency period. With good monsoon rains this year, this project will potentially have benefited many thousands of people. The Federation Secretariat decided in May 2001 that the evaluation of the India Earthquake Operation should be divided into two stages, a formative inception stage that was completed in September 2001 and the main evaluation to which this TOR refers. The two-stage process was adopted because of a perceived need to first create a clear focus for the evaluation without which the potentially broad scope of the evaluation might have made it unmanageable. The principal output of the Inception Report was this Terms of Reference, which has been designed to prioritise those areas where lesson learning will have the greatest impact on the future performance of the Federation in emergency response and rehabilitation. The Inception report also provides the evaluation team with substantial background information on each of the evaluation s areas of focus. Purpose The evaluation has two main aims. These are to examine a) the impact and quality of the Federation emergency operation in Gujarat from the beneficiaries viewpoint and b) aspects of management and coordination key to Page i

17 improving the Federation s performance in both disaster response and rehabilitation planning, in the light of its experience in the Gujarat operation. Objectives The evaluation will undertake analysis, develop findings and make recommendations in the following areas: Impact and Quality of the Federation s Disaster Response The evaluation will review the impact of Federation s provision of goods and services against the following criteria: Assessment of need Coverage achieved in meeting the needs of the most vulnerable Effect on the condition and livelihoods of beneficiaries Qualitative aspects of appropriateness, relevance, dignity, fairness and equity in the provision of goods and services will be assessed against the Federation s formally adopted criteria for disaster response: Principles of Conduct for The International Red Cross and Red Crescent Movement and NGOs in Disaster Response Programmes, 1994 Sphere Humanitarian Charter and Minimum Standards, 2000 The Federation s Emergency Response Policy, 1991 The evaluation will comment on the relevance of the codes and policies to the India earthquake context. A primary source for impact and quality assessment will be the beneficiary survey see under Methodology. Transition to Rehabilitation The evaluation will investigate the reasons behind the Federation s apparent loss of momentum and lack of coordination in moving to a plan and Appeal for rehabilitation and make recommendations for assessment, planning, coordination and funding mechanisms that will improve the Federation s performance in future emergencies in India and elsewhere. The evaluation will pay particular attention to the models of cooperation and coordination between National Societies and the coordination role of the Secretariat. The planning of the rehabilitation phase will be assessed against the Federation s Post-Emergency Rehabilitation Policy, Capacity Building of the India Red Cross Society The evaluation will assess the impact of the India Earthquake Operation on the India Red Cross capacity to respond to other disasters in Gujarat and elsewhere in India, and the potential capacity development from the planned rehabilitation programmes, especially those for disaster preparedness. The evaluation will make recommendations as necessary for changes in the capacity building approaches of the Indian Red Cross, the Secretariat, and the National Societies. Federation Management Systems The evaluation will focus on specific aspects of the Federation s management of its disaster response. Using the India Earthquake Operation as a case study, the evaluation will review: The use of surplus emergency funds, including the inclusion of construction programmes in the rehabilitation appeal, and make recommendations as necessary on alternative mechanisms for the use of such funds, taking into account donor requirements Respective roles of Disaster Management and Coordination Division and Programme Coordination Divisions in disaster response and rehabilitation planning Alternative methods of, and recommendations for, the temporary addition of human resources at key points of need during disaster response, including in Geneva and Secretariat regional offices, and taking into account the scope for National Societies to strengthen Secretariat operational management capacity on a temporary basis Page ii