Meyu Muluke woreda, ETHIOPIA July 19 th to 29 th 2013 Inés ZUZA SANTACILIA

Transcription

1 Meyu Muluke woreda, ETHIOPIA July 19 th to 29 th 2013 Inés ZUZA SANTACILIA

2 ACKNOWLEDGEMENTS International Medical Corps (IMC) and Coverage Monitoring Network extend its deep gratitude to all those who have contributed to this study including: the authorities in Meyu Muluke woreda (district) and East Hararghe province (Oromia Region), Ethiopia and to all the health personnel and village residents for their hospitality and cooperation. Avery special thanks to the mothers and caregivers of severely acute malnourished children. A very special thanks to the IMC team in Harar, to the Nutrition Officer (ALMAZ TASISA) for his support coordinating the SQUEAC at field level and for his contribution on the improvement of this report. Thanks you to the IMC team in Addis Ababa; the National Nutrition Manager-IMC (Beka TESHOME) for his collaboration. Thank you to the Federal Ministry of Health (MoH) for their zeal, support and motivation. The East Hararghe Nutrition Focal Person (Mesfin WORKU) for his support during the Harar training. To the Nutrition Focal Person (Daniel SISAY), the Extended Program of Immunization Focal Person (Iskender Mohamed), the Nutritional Survey Field Worker (Abdulahi AHMED) and the TB/HIV Focal Person (Ashenafi DOLEBO) in Meyu Muluke woreda. Thank you also to Ezana TESFAYE ZEMO from the Autralian MoH for all his support and commitment with the SQUEAC. This study would not have been possible without the hard work and commitment of everyone involved. Lastly, thank you to the Office of Foreign Disaster Assistance (OFDA) through GOAL for financing this project. 2

3 EXECUTIVE SUMMARY Meyu Muluke woreda (district) is one of the 19 woredas in East Hararghe province in the Oromia Zone of Ethiopia. It is composed of approximately 124 villages (divided into 19 kebeles) and an estimated population of 54,496 inhabitants (14.8 % of children between 6 and 59 months). The Dinkas and Luo tribes are the predominant ethnic group inhabiting Meyu Muluke woreda. Communities are pastoralist and agro pastoralist who continue to face food insecurity from re-occurring drought and subsequent livestock losses. International Medical Corps (IMC) was running a program to contribute to the reduction of morbidity and mortality related to acute malnutrition and improves nutrition practices in three woredas (Meyu Muluke, Kumbi, and Fedis) in East Hararghe Zone from January to July IMC has been working in food-insecure woredas of East Haraghe Zone since 2005, responding to the emergency nutrition needs. In Ethiopia, the nutrition services are delivered by the MoH. In July 2013 there were 13 OTP sites functioning in the Meyu Muluke woreda. The other six OTPs were having security problems. There are also three stabilization centres. Since the beginning of 2012 the MoH has created the Health Development Army (HDA). It is a team of community-level volunteers engaged in screening and mobilizing children under 5 and pregnant and lactating women. They can detect cases of a defined number of diseases (including malnutrition) and refer them to the health facilities. There is one HDA per 4 households. The Health Extensions Workers are formal salaried workers within the health system. They provide treatment for Severe Acute Malnutrition (SAM) as part of a Health Extension Program. Regarding the nutritional situation, no data of Meyu Muluke woreda is was available for the previous years. But data from Midega Tolla woreda, the nearest woreda available data (East Hararge Zone) in November 2012 was available the Global Acute Malnutrition and SAM rates which were respectively 10.0 % ( % C.I.) and 0.2 % ( % C.I.). Resume of coverage assessment The coverage assessment was conducted to evaluate access and coverage of the Community based Management of Acute Malnutrition programme for children aged 6 to 59 months with SAM. It was conducted between July 19 th and 29 th 2013 and it was the first of its kind for the area. It was conducted at the beginning of the rainy season and the Ramadan. 3

4 The SQUEAC methodology used consisted of 3 stages, applying the principles of triangulation (by source and method) and sampling to redundancy. The coverage investigation conducted in Meyu Muluke woreda showed a period coverage of 90.5% (95% IC: 81.6% %) The table below presents the main barriers on which the program must act to improve coverage as well as specific recommendations how to do so: Barriers - Long distances - Previous Rejection - Stigma (mother ashamed) - Wrong Admission and Discharge Criteria (MUAC at discharge < 110) - Insecurity Recommendations 1. Advocacy on including the WHO standards in the national guidelines 2. Implement mechanism to reduce impact of the distance and inaccessibility 3. Strengthen community sensitization 4. Reinforce supervision and improve data quality and follow up 5. Think over IMC strategy during the periods of no programme implementation / sustainability 6. Repeat the SQUEAC in six months or one year // before IMC support 4

5 CONTENTS 1. INTRODUCTION CONTEXT International Medical Corps in Meyu Muluke woreda OBJECTIVES METHODOLOGY GENERAL OVERVIEW STAGES ORGANIZATION OF THE EVALUATION LIMITATIONS RESULTS STAGE Quantitative data analysis Qualitative data analysis STAGE STAGE A. The prior B. The likelihood C. The posterior DISCUSION RECOMMENDATIONS Annex 1 : Survey questionnaire for current SAM children NOT in the program Annex 2: Meyu Muluke wereda SQUEAC plan, July Annex 3 : SQUEAC Survey team Annex 4 : Terminology in Oromifa used to describe malnutrition and RUTF. Meyu Muluke woreda. Ethiopia. SQUEAC July Annex 5: Weighted BBQ, Meyu Muluke woreda SQUEAC, Ethiopia. July

6 ABBREVIATIONS BBQ CI CHD CMAM CMN CSAS EPI FDA GAM HC HDA HF HP IMC INGO LoS LP MAM MoH MUAC ODPPC OFDA OTP RHB RUTF SAM SC SFP SSI SQUEAC TSFP UNICEF WHO Barriers, Boosters and Questions Credible Interval County Health Department Community based Management of Acute Malnutrition Coverage Monitoring Network Centric Systematic Area Sampling Expanded Program on Immunization Food Distribution Agents Global Acute Malnutrition Health Centers Health Development Army Health Facility Health Post International Medical Corps International Non-Governmental Organisation Length of Stay Land Preparation Moderate Acute Malnutrition Ministry of Health Mid-Upper Arm Circumference Oromia Disaster Prevention and Preparedness Commission Office of Foreign Disaster Assistance Outpatient Therapeutic Programme Regional Health Bureau Ready to Use Therapeutic Food Severe Acute Malnutrition Stabilization Centre Supplementary Feeding Program Semi Structure Interview Semi Quantitative Evaluation of Access and Coverage Targeted Supplementary Feeding Programmes United Nations Children s Fund World Health Organisation 6

7 1. INTRODUCTION 1.1 CONTEXT Overview of the area Meyu Muluke woreda (district) is one of the 19 woredas in East Hararghe province in the Oromia Zone (region with 180 woredas) of Ethiopia. Meyu Muluke woreda is composed of approximately 124 villages (divided into 19 kebeles) and an estimated population of 54,496 inhabitants 1. With 14.8 % of children between 6 and 59 months (8,087 children). The Dinkas and Luo tribes are the predominant ethnic group inhabiting in the woreda. The altitude of this woreda ranges from meters above sea level. Figure 1: Ethiopia, Oromia Zone and East Hararghe province 2. East Hararghe province Meyu Muluke woreda lies between ' and ' N latitude and ' and ' E longitude to the south of Harar town. It is bordered by Girawa and Bedeno woredas to the North, Gola Oda woreda to the West, Fedis woreda to the East and Bale zone and Somali regional state to the south. 1 From woreda health office. 2 From wikipedia : (visited on September 2013) 7

8 The woreda has a total area of 4, Km 2 accounting for about 22.02% of the total area of East Hararghe Zone. It is located at a distance of 150km from Harar town to the south direction. There are two rainy seasons in the area, meher (June-September), used for crop production, pasture and water harvest and the short belg rains (February- May), mainly used for land preparation, planting of long cycle crops collected after the meher rains, small scale production, and improving water and pastures. While central and western parts of Oromia Region generally received normal to abovenormal rains during the kiremt season this year, drought-prone areas in the east including East Hararghe Zone received insufficient rainfall; inadequate for crop development. The communities living in Meyu Muluke woreda are pastoralist and agro pastoralist who continue to face food insecurity from re-occurring drought and subsequent livestock losses Nutritional situation Regarding the nutritional situation, International Medical Corps (IMC) in conjunction with Region s Oromia Disaster Prevention and Preparedness Commission (ODPPC) has conducted a Nutrition and retrospective mortality survey (SMART survey 3 ) in Midega Tolla woreda of East Hararge Zone in November 2012 (post-harvest periods). The Global Acute Malnutrition (GAM) and Severe Acute Malnutrition (SAM) rates were 10.0 % ( % C.I.) and 0.2 % ( % C.I.). No national data of Meyu Muluke woreda is available for the previous years. Currently in Oromia Regional State, with the exception of a few areas (where NGOs carry out repeated nutrition surveys), there is no nutrition surveillance /no time series data on nutritional status to inform program planning and management. Just to fill this gap ODPPC has planned to undertake nutrition surveillances together with (RHB). There are various ways to implement nutrition surveillance; these include growth monitoring programs, longitudinal anthropometric data systems, community or institution-based sentinel sites systems and repeated cross-sectional surveys. For startup, the region has conducting six cross-sectional surveys that expected to be repeated bi-annually with the support of United Nations Children s Fund (UNICEF) in six woredas. Midega Tolla is one of the targeted woreda for the above mentioned purposes 4. An OTP coverage assessment using the Centric Systematic Area Sampling (CSAS) methodology was conducted in November-December 2010 in Babile woreda, Oromia 3 based on NCHS growth reference IMC and ODPPC. Nutrition and retrospective mortality survey. November

9 region by IMC. The assessment showed an overall OTP period coverage of 55.3% (95% CI: 46.2% %) and OTP point coverage of 46.0% (95% CI: 38.7% %). No coverage assessment has been done in Meyu Muluke woreda Health access in Meyu Muluke woreda Services are delivered both at Health Posts (HP), which are closer to community level and deliver primary health care and at Health Centers (HC), which are located within three kebele of Meyu Muluke woreda (Husse, Alola and Chella kebele). Since the beginning of 2012 the MoH has created the Health Development Army (HDA). It is a team of community-level volunteers engaged in screening and mobilizing children under 5 and pregnant and lactating women. They can detect cases of a defined number of diseases (including malnutrition) and refer them to the health facilities. There is one HDA per 4 households. The community has chosen them with the participation of the MoH. In many cases they have included traditional healers or traditional birth attendants into this HDA or community base volunteers (communitylevel volunteers previously working with IMC). Nevertheless not all of them are trained. They have attended discussion in HF about some diseases (especially in children). Mostly of them do not have MUAC tapes. Only the ones that were previously community base volunteers with IMC have it. The Health Extensions Workers (HEWs) are formal salaried workers within the health system (in HP). They provide treatment for SAM as part of a Health Extension Program containing 16 packages on topics such as hygiene, family health, disease prevention and control, and health education. Each HEW supervises along of HDA. In Meyu Muluke woreda except insecurity area all kebeles have HP and at each HP there are two health extensions workers (HEWs), so in Meyu Muluke woreda there are 13 HP and 3 health centers (HC). 9

10 Nutrition services 5 In Ethiopia, the nutrition services are delivered by the MoH. In July 2013 initially there were 13 OTP sites functioning in the Meyu Muluke woreda. Seven OTPs were inaccessible due to security problems (one of them became insecure during the investigation). Finally only 12 OTP sites were functioning in Meyu Muluke woreda at the end of the month. There were two HEWs workers per HP. They have been first health assistants and after one year training became HEW. There are also three stabilization centres (SC) in Meyu Muluke woreda: Husse SC, Alola SC and Chella SC Since 2004, the MoH started to integrate the in-patient and out-patient management of severe acute malnutrition into hospitals and HC (i.e., at regional and woreda levels). In 2008, the out-patient management of SAM was further decentralized to HP (i.e., at the kebele level). The objective was to ensure access to and coverage of malnutrition services by bringing the service closer to the community. It benefits families by reducing opportunity costs of accessing treatment. It also benefits the health system through capacity building and acts as the catalyst for strengthening nutrition activities within health facilities and at the community level, for treatment and prevention of malnutrition. The programme is in line with the first component of the National Nutrition Programme (NNP), with its focus on Supporting Service Delivery which includes Increased Access for the Management of SAM. The Federal MoH Protocol for the management of SAM is from The admission criteria are Weigh for Height < 70% (or <-3 Z-score using the WHO-2006 standards), MUAC < 110 mm with (with length > 65 cm) or presence of bilateral pitting oedema. In Meyu Muluke woreda the admission criteria is based on national protocol for SAM management. At HP level the activity implemented by health extension worker where the admission criteria is MUAC <110mm(with length >65cm) and/or bilateral pitting oedema. At HC level the activity implemented by nurse or health officer where the admission criteria is Weigh for Height < 70% (or <-3 Z-score using the WHO-2005 standards), MUAC < 110 mm with (with length > 65 cm) and/ or presence of bilateral pitting oedema. UNICEF provides the Ready to Use Therapeutic Food (RUTF) and medicines for SAM treatment. 5 IMC. Therapeutic Feeding Programme Coverage Assessment Report. Babile Woreda, Oromiya Region, Ethiopia December

11 For Moderate Acute Malnutrition (MAM) management, International Medical Corps manages cases of MAM in collaboration with the World Food Program (WFP), DPPC office, health offices and the community. Supplementary food was provided by the WFP. Based on discussions with the WFP and DPPC during implementation, International Medical Corps managed Extended Outreach Service (EOS) beneficiaries (beneficiaries who were identified by MoH mass screening each three month (Child Health Day (CHD) screening). Including also non-eos beneficiaries who were newly identified by International Medical corps through the Targeted Supplementary Feeding Programmes (TSFP) staff screenings, MAM beneficiary and children graduated from the OTP. Most of the time delay of SFP food after screening is commonly due to transportation issue from government. SFP food is distributed by International Medical Corps SFP staff and Food Distribution Agents (FDA). 1.2 International Medical Corps in Meyu Muluke woreda International Medical Corps (IMC) was running a program to contribute to the reduction of morbidity and mortality related to acute malnutrition and improve nutrition practices in three woredas (Meyu Muluke, Kumbi, and Fedis) in East Hararghe Zone from January to July IMC has been working in food-insecure woredas of East Haraghe Zone since 2005, responding to the emergency nutrition needs caused by recurring failed seasonal rains, which have negatively impacted the nutritional well-being, food security, and general health status of the population. Between emergency nutrition programs there are usually funding/support gaps. More recently until October 2012, IMC, with funding from the Office of Foreign Disaster Assistance (OFDA) through GOAL has been implementing the Emergency Nutrition Support Program in four woredas (Midega Tolla, Meyu Muluke, Kumbi and Gursum) to respond to the nutrition crisis and reduce the disaster risk for targeted populations using the community-based management of acute malnutrition (CMAM) approach. This programme included capacity building of Ethiopian Ministry of Health and other stakeholders through technical training, medical supply and equipment provision and nutrition education to improve responsiveness and promote behavioral change during the programme. The support from January to July 2013 was composed of the four components of the CMAM approach including: 1) Targeted Supplementary Feeding Program (TSFP) for MAM, 2) SFP for MAM pregnant and lactating women and other vulnerable groups, 3) Outpatient Therapeutic Program (OTP) and Stabilization Centers (SC) for SAM and 4) community mobilization and outreach activities. 11

12 At OTP/SC level IMC build the capacity of MoH staff and conduct close supportive supervision to ensure the quality of the service. UNICEF has provided the RUTF to MoH and IMC has provided logistical support to transport the RUTF from the zonal warehouse to woreda stores and from woreda stores to the health facilities. HEWs and health center staffs have managed the malnourished children at health posts and health centers. The emergency nutrition program is implemented in close collaboration with the MoH, DPPC, UNICEF, WFP as well as other stakeholders. 12

13 2. OBJECTIVES Main objective The main objective of this study was to evaluate access and coverage of the Community based Management of Acute Malnutrition (CMAM) for children aged 6 to 59 months with SAM in Meyu Muluke woreda, in East Hararghe province in the Oromia Zone of Ethiopia, using the Semi-quantitative evaluation of access and coverage (SQUEAC) methodology. Specific objectives - To develop capacity of various stakeholders on undertaking program coverage assessments using SQUEAC methodology - To determine baseline coverage for CMAM - To identify boosters and barriers influencing CMAM program access and coverage - To develop feasible recommendations to improve CMAM program access and coverage Photo 2 : Stage one training at Harar town (Winta Hotel) for data collection, Ethiopia. 13

14 3. METHODOLOGY GENERAL OVERVIEW The Semi-Quantitative Evaluation of Access and Coverage (SQUEAC) is a coverage assessment method developed by Valid International, FHI 360/FANTA, UNICEF, Concern Worldwide, World Vision International, Action Against Hunger, Tufts University, and Brixton Health. The methodology is semi-qualitative in nature, meaning that it draws from a mixture of both quantitative data from routine program monitoring activities as well as qualitative data collected on the field. This mixed methods approach combines data sources to estimate program coverage and to develop practical measures that can improve access and coverage. - Quantitative data came mainly from routine monitoring information that the program already collected including: admissions, defaulting, recovery, middle upper arm circumference (MUAC). Routine program data was coupled with complementary data like agriculture, labor, and disease calendars, anthropometric nutritional surveys, and agricultural and food security assessments. - Qualitative data collected came from interviews, focus groups and questionnaires with various key informants. Together, the data were triangulated by source and method to formulate hypotheses about coverage and access. Data triangulation is a powerful technique that helped validate our findings through cross verification. Hypotheses were then tested with small-area surveys and small sample surveys. Then, a wide area survey was conducted in the community to determine the point coverage estimate. Lastly, the results from the quantitative and qualitative analyses and the wide-area likelihood survey were combined and the overall global coverage estimate was calculated using Bayesian statistical techniques SQUEAC and SLEAC Technical Reference. FANTA. Available at Oct2012_0.pdf 14

15 The coverage study was conducted between July 19 th and 29 th It was the first of its kind for the area for SAM. It was conducted at the beginning of the rainy season and the Ramadan. The SQUEAC methodology used consisted of 3 stages, applying the principles of triangulation (by source and method) and sampling to redundancy STAGES Stage 1: Identification of potential areas of high and low coverage and access barriers Identification of potential areas of high and low coverage using routine program data; in this stage, triangulation of data was done by various sources and methods as highlighted below. 1. Quantitative data (February June 2013) Quantitative, routine program data helped to evaluate the general quality of CMAM service, to identify admission and performance trends and to determine if the program adequately responds to need. It also helped point out problems in screening and admission. Lastly, routine program data analysis provided the first insights into variation in program performance between OTPs. Routine program data analysis included the following (for 13 OTP) - Global (OTP and SC) trends of admission and defaulters over time and compared to the agricultural calendar, the lean period, child epidemics and diseases, workload, weather patterns and key events - Admission: admission by OTP and SC - OTP and SC program performance indicators over time (recovery, default, death, non-response). - Discharged o Cured: length of stay (LoS) and MUAC at discharge. - Stock break out data. Complementary data from children card (for 11 or 12 OTP 7 ) - MUAC at the time of admission (12 OTP) - Length of Stay for discharged cured (11 OTP) 7 Mojo weldia OTP area started to have security problems during the SQUEAC in July 2013 and some data could not be obtained. Goro Neyeda OTP data for length of stay could not being colected. 15

16 - The village lists populations belonging to each OTP and distance to OTP. Admissions per village Defaulter: there were no reported defaulters in the period, Not available - Admissions and other OTP indicators previous to February Qualitative data Qualitative data was collected to investigate program operations, to unravel the opinions and experiences of personnel involved in CMAM and to identify any potential barriers to access. The following methods were used: focus groups, semi-structured interviews, structured interviews, case studies and observation. Interviews and focus groups were conducted with key informants either directly or indirectly involved in the CMAM program. These included: women s and men s community, program personnel of IMC, local authorities (not religious leaders due to the Ramadan period) HDA, HEWs, caregivers of SAM children, health authorities. Informal caregivers (traditional healers and traditional birth attendants) were not found because mostly were integrated in the HDA. The BBQ framework. Throughout the investigation, the data are going to be organized, analyzed and triangulated using the Barriers, Boosters and Questions (BBQ 8 ) framework. It is a tool that facilitates iterative data collection that is then categorized into one of three categories. The various data organized within the BBQ framework, when combined, will help providing information about where coverage is likely to be satisfactory as well as where it is likely to be unsatisfactory. Additionally, the BBQ provided information about likely barriers to services access that exists within the CMAM program. Stage 2: Confirms the location of areas of high and low coverage The goal of stage 2 is to test the hypotheses about coverage and access elaborated in stage 1. These hypotheses usually take the form of identifying areas where the combined data suggest that coverage is likely to be either high or low. The small-area surveys method was used to test the hypotheses for CMAM high and low coverage areas. 8 Barriers are negative findings that deter from program coverage and complicate access to service. Conversely, boosters contribute to a higher coverage and facilitate access. Lastly, questions, are those findings elements to be further investigated, and either become a barrier or booster or remain inconclusive 16

17 The active and adaptive case-finding methodology was used to find SAM cases. Data surveys will be analysed using simplified lot quality assurance sampling (LQAS). The LQAS classification technique analyses data using the following formula: where the threshold value number of cases found coverage standard If the number of covered cases found (that is, those cases in the program) is greater than then then the coverage of the surveyed area is classified as being greater than or equal to the coverage standard. If the number of covered cases found (that is, those cases in the program) is less than then then the coverage of the surveyed area is classified as being less than or equal to the coverage standard The threshold chosen is 40%. The Centric Systematic Area Sampling (CSAS) coverage survey done in Babile woreda in November 24 th to 5 th of December 2011 by IMC was the guide to establish this threshold. If the number of covered cases found (that is, those cases in the program) is less than then then the coverage of the surveyed area is classified as being less than or equal to the coverage standard. 17

18 Stage 3: Wide area survey conducted to estimate overall coverage. The goal of stage three is to calculate the overall coverage estimate. This is done using a Bayesian statistical technique called beta-binomial conjugate analysis. Conjugate analysis begins with a beta distributed, probability density called the prior. The prior is then combined with a binomial distributed, likelihood function called the likelihood. The likelihood is going to be determined by a wide-area coverage survey that will be conducted across the entire program catchment area; the mode of the likelihood was, in fact, the point coverage estimate from the survey. Because the prior and the likelihood are mathematically expressed in similar ways (as probability distributions) they can be combined through conjugate analysis, the result of which is the posterior probability density the posterior. The mode of the posterior is the final coverage estimate. 1. The Prior The prior was constructed by combining the results from stages 1 and 2, that is: routine program data, qualitative data and all relevant findings from the small-area and small sample surveys. The prior was the result of combining two modes: 1) The weighted BBQ : a score from 1 to 5 was attributed to each element. The score reflected the relative importance or likely effect that the element had on coverage. The coverage estimate was calculated by the method explained above. 2) The histogram prior : During a participatory working group, the investigation team designed a histogram representing the prior mode. This was done realistically and democratically. The mode, minimum and maximum coverage values were chosen credibly. 2. The likelihood A wide-area likelihood survey was conducted over the entire program catchment area to calculate the coverage estimate. The active and adaptive case-finding methodology was used to identify the SAM cases. The case definition used for the coverage survey was defined as a child matching the admission criteria of the programme. The admission criteria of the Ethiopian CMAM programme included children aged between 6 and 59 months with at least one of the following criteria: 1) a MUAC of <11.0 cm and 2) bilateral pitting oedema A simple structured interview questionnaire was used to caregivers of non-covered cases for SAM in Annex 1.

19 The sample size required was calculated by using the following equation:. ( ) 1. Mode: prior value expressed as a proportion. 2. α et β: shape parameters of the prior. 3. Precision: desired precision. In the present case the precision used was 0.14 (14%). 4. SAM prevalence: 0.5% was chosen after stage 2 results to be the possible prevalence in the area. Because no available data of Meyu Muluke wereda was available. In the near Midhaga Tolla woreda of East Hararge Zone in November 2012 the SAM rate was at 0.2 % ( % C.I.). 5. Average village population: 401 people in Meyu Muluke woreda (based on woreda health office data) 6. Population between 6 and 59 months : approximately 14.8% And the minimum number of villages needing to be sampled to achieve the sample size was calculated using the following equation: X The number of required villages was randomly selected with ENA for SMART software 9 from the list of accessible villages in Meyu Muluke woreda. 3. Overall Coverage Estimate The point or period coverage estimate was chosen for SAM coverage. By method of Bayesian betabinomial conjugate analysis the prior probability density was combined with the coverage estimate from the likelihood survey to calculate the mode of posterior probability density. The Posterior Probability is the estimate of the overall coverage: it represents the synthesis of the prior probability and likelihood generated by the calculator with Bayes credible interval (CI) of 95%. Recommendations and Action Plan: A final important step is the development of an action plan that clearly identifies the actions to be undertaken, indicators, evaluation methods and deadlines. 9 Available at: [Accessed: November 2013] 19

20 3.3. ORGANIZATION OF THE EVALUATION CMN technical support The IMC team and the Ethiopian MoH from East Hararghe province received the technical support of the Coverage Monitoring Network (CMN). The CMN Project is a joint initiative by ACF, Save the Children, International Medical Corps, Concern Worldwide, Helen Keller International and Valid International. The programme is funded by ECHO and USAID. This project aims to increase and improve coverage monitoring of the CMAM programme globally and build capacities of national and international nutrition professionals; in particular across the West, Central, East & Southern African countries where the CMAM approach is used to treat acute malnutrition. It also aims to identify, analyse and share lessons learned to improve the CMAM policy and practice across the areas with a high prevalence of acute malnutrition. The technical and methodological support was provided by a Regional Coverage Advisor (RECO) Inés ZUZA SANTACILIA. During the evaluation CMN support was conducted in three phases: - 1st phase: remote technical support for the planning and preparation of the evaluation with the CMN RECO. - 2nd phase: in field technical support in Meyu Muluke woreda. The CMN RECO was deployed to support training on the use of the SQUEAC methodology and the implementation of the evaluation until stage rd phase: remote support for the completion of the investigation, analysis of results and report writing. SQUEAC plan in Annex Team training, logistic organization and evaluation development The investigation team (described in Annex 3) was composed of members of IMC from Harar and Addis Ababa team, MoH staff (from East Hararghe province and Meyu Muluke woreda), one partner (MoH of Australia) and one nutritional survey field worker recruited for the SQUEAC. The SQUEAC was conducted in the field by the CMN RECO in collaboration with the Monitoring and the Harar Nutrition Officer (AIMAZ TASISA). A two days training in the SQUEAC methodology was made by the CMN RECO in Harar. This training targeted people that integrated the evaluation team and other people who might be interested in the methodology. The East Hararghe Nutrition Focal Person from the Ethiopian MoH participated on this two days training. After the team was deployed to Meyu Muluke woreda (150km away from Harar) in the mountains. Some of the investigation team members stayed in the Husse capital of Meyu Muluke woreda while the RECO and the other members of the investigation team were based in Girawa (70 km from Harar 20

21 in Meyu Muluke woreda). This was due to the security and the logistic conditions in Meyu Muluke woreda. During the SQUEAC all the investigation team was also deployed to Girawa (to avoid daily travel from Meyu Muluke to Girawa) For the three steps the investigation team was divided in five teams, composed by normally two people each LIMITATIONS The evaluation was limited by the following elements: - The security situation didn t allow accessing one OTP and its area (of the 13 functioning). - The mountain roads and bad weather conditions (rain and fog) made the logistical coordination, communication and deployment of the teams difficult. - There was no telephone network in Meyu Muluke woreda. In Giraw, the telephone network had problems (frequent cut-offs) and no internet was available in both areas. The combination of these factors made impossible to share daily information from all the teams involved in the SQUEAC. Appointments in Girawa or Meyu Muluke woredas were done for sharing the information and continue the training. Also the RECO was unable to communicate and give technical support to the investigation team between the beginnings of stage 2 until the finalization of the SQUEAC (when the Harar Nutrition Officer was travelling back to Harar). - Initially technical support from National Nutrition Manager was going to be given for the investigation team along all the SQUEAC. Finally it could not go more than stage 2. - No data was available from the IMC programme or the MoH from before February 2013 for the SQUEAC. Photo 3: Training on stage two at Girawa woreda. 21

22 4. RESULTS 4.1. STAGE Quantitative data analysis a. Needs response : admissions and defaulters trends compared to seasonal and key events calendar Figure number 2 shows the OTP admission over a 4-month period (February June 2013). This graph is aligned with a seasonal and key event calendar developed by the investigation team (weather patterns, seasonal calendar of human diseases associated with SAM in children, food availability, and workload). Together these two figures helped evaluate to what extent the program responds to seasonal needs. There were no defaulters along these months. The Ethiopian calendar (Ge'ez Calendar 10 ) starts in from the 29 th August. The period showed in figure 2 corresponds to the Ge ez calendar 2005 year. Since January to June 2013, 601 SAM children have been admitted to OTPs with a mean of children admitted per month. Zero defaulters were notified during the period. Data quality issues were detected in one OTP along the register revision in Stage 1. The SAM admission trends are reflecting few months of the year trends. The hunger gap is from December to March, with a peak in February. Nevertheless, trends of admissions show the increase in the number of cases along the 4-month period. Comparison with the trends of admissions in other period should be necessary to extract reliable conclusion of these data. Normally however, the combination of prone diarrhea, food prices and hunger gap should make February the peak month for admissions. However, water shortage and displacement of families looking for water have made OTPs less accessible for the community. 10 The Ge'ez Calendar is the official calendar in Ethiopia. It is based on the Coptic calendar with a leap day, every four years. The Ethiopian Calendar has twelve months with 30 days each and a thirteenth month called Pagume with five or six days depending on the year. 22

23 Number of SAM cases admitted Figure 2. OTP admission patterns over time compared with seasonal event calendar, Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June Fev March April May June Hot Seasonality S O N D J F M A M J J A Dry Dry bit rain Dry Rainy season * ** *** spring summer autumn winter Birra Bonna Afrassa Gana Hunber gap Diarrhea ARI Malaria* Food Price Activities LP** Weeding Grazing Harvest LP Sowing Water shortage Travel*** (looking for Water) Mass Screening *Half of the Woreda (around eight Kebeles) are malaria endemic areas. The other areas are less endemic. In April-May there are areas where is very prone. June-August is prone for all the places. ** LP: Land Preparation *** The whole family displaces to look for water, especially in six Kebeles. It happens every year. b. OTP vs. SC admissions The percentage of children admitted to the SC could be an indicator of the timeliness of admissions. It is directly related to the percentage of SAM cases that arrive at the OTP with associated medical 23

24 Number of cases complications. Children remaining untreated for long periods with declining nutritional status develop medical complications and end up needing SC care. A high percentage of SAM cases with medical complications could often be the product of a late presentation and uptake of services. In Meyu Muluke woreda, the proportion of program admissions requiring inpatient care from February to June 2013 is only 4.8%. This percentage is less than the 5% recommended for established programs therefore this can indicate early admission of SAM children in OTP services. Figure 3. OTP admission compared with SC admissions. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June % 95.2% Admissions OTP Admission SC c. Admissions by OTP Figure 4 shows the number of SAM cases admitted per OTP over the 4-month period (February June 2013). Alalo OTP is the one that received more cases during the period (161 SAM admissions). Figure 4 : SAM admissions per OTP site. Meyu Muluke woreda, Oromia region, East Hararghe province, Ethiopia. February-June OTP 24

25 Percentage Alalo OTP is the nearest OTP to the areas that have insecurity problems, in the border (with no OTP functioning in the area). Go Sodoma OTP however is the one that received the fewer number of cases (20 SAM admissions). IMC team explained that this can be related to the river passing through the kebele that is used for irrigation therefore allowing the community to be less food insecure. Figure 5 shows the percentage of SAM cases admitted per OTP and the percentage of population of the catchment area per OTP over the 4-month period (February June 2013). Figure 5: Percentage of SAM admissions per OTP and percentage of population catchment area. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June % 25% 20% 15% 10% 5% 0% Percentage population Percentage admissions OTP Alalo OTP is the one that received proportionally much more percentage of cases than expected for their catchment area. Husse OTP has also more percentage of admissions that percentage of population in the health area. In the opposite, Gebibeda and Mojo Weldia OTP have a lower percentage of admissions compared to their percentage of population in the catchment area. d. Admissions MUAC Admission MUAC is an indicator for late /early presentation and service uptake at the OTP level. It can be a measure of direct coverage failure because late admissions are those non-covered SAM cases that went untreated for a significant period of time. Late admissions almost always require inpatient care and are associated with prolonged treatment, defaulting and poor outcomes. Figure 6 reports the MUAC distribution for SAM cases admitted by MUAC from February to June The admission MUAC criteria is < 110 mm. The MUAC median at admission was 107 mm (in red). That means 50% of the children arrive with a MUAC less than 107 mm. 97 mm is the inferior value for MUAC admission along the period. This can indicate early admission/detection of SAM children in the OTP programme. Yet there is still room for improvement because many cases have been admitted with MUAC < 105 mm (with very high risk of mortality). The median at admission by in general is very similar in all OTP. The OTP with less median MUAC at admissions were Biko and Goro Negeya (105 mm). And the one that had the better median MUAC was Gebibeda OTP with 109 mm. 25

26 Number of cases During the analysis of MUAC data, an over-representation of rounded values (i.e. 105 mm, 100 mm, etc.) was observed, indicating imprecision in the MUAC measurement. Figure 7: MUAC at OTP admission. Meyu Muluke woreda, East Hararghe province, Ethiopia. February- June median MUAC (mm) e. Admission by type In Meyu Muluke woreda admissions are based on the presence of MUAC < 110 mm with (with length > 65 cm) or presence of bilateral pitting edema. In Meyu Muluke Woreda OTP 27.6% of the admissions were done by bilateral pitting edema (for the period of February to June 2013). These results will need further investigation to understand better the reasons for this high percentage of admissions by edema 11. Looking in figure 8 at the percentage of admissions per OTP there are differences. Chella and Borka Jenta OTP had 46.9% and 40.0% of admissions by edema respectively while Chira OTP had 8.1% of admissions. 11 More information about Kwashiorkor: Briednd A; Myatt M; Dent N; Brown R. Putting kwashiorkor on the map. CMAM Forum. Available at URL: CMAM-Forum-2013.pdf [Accessed: November 2013] 26

27 Percentage Figure 8: Percentage of admissions by bilateral pitting edema per OTP admission. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June % 40% 30% 20% 10% 0% OTP f. Performance indicators The performance indicators per OTP over the 4-month period (February June 2013) indicate that 100% of children were cured, The performance indicators for the two SC showed that 100% of children were stabilized along the period. g. Discharged cured The length of stay before recovery provides helpful insight into the duration of the treatment episode (e.g. the time from admission to discharge). In figure 9 the OTP length of stay (LoS) for 12 OTPs (February to June 2013) has a median duration of 6 weeks. Usually international standards define typical LoS may be days (4 to 6 weeks) and of maximum 8 weeks. In this case the maximum length of stay was 10 weeks, with 97.5% of the cases having 8 weeks of stay. 27

28 Percentage Number of children Figure 9: Length of stay for discharge cured. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June median weeks Figure 10 shows the percentage of MUAC at discharged cured under 110 mm (in 12 OTP) for the period February-June In general 13.9% of the discharged cured (65 cases) had MUAC at discharge < 110 mm. From them 41.5% had 109 mm, 29,2% 108 mm and 3.1% 107 mm. This could indicate early discharge of SAM children from the OTP programme. The reasons for it should be studied to try to avoid this situation. It can be observed that Borka Jenta had 54.3% of discharged cured with MUAC < 110 mm. Some OTP had more than 20.0% of the discharged cured with MUAC < 110 mm (Goro Negeya, Ifabase and Gugufe OTP). While others in almost all cases the discharge criteria were above 110 mm (Goro Sodoma and Alalo). Figure 10: Percentage of SAM children discharged cured with MUAC < 110 mm. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June % 50.00% 40.00% 30.00% 20.00% 10.00%.00% OTP 28

29 Percentage Distance influence on OTP admissions Figure 11 reports the percentage of admissions compared to the percentage of population per Km of distance to the OTP (January 2012 Mai 2013) at 5 weeks. It is appreciated that most of the population live in the 5 km range around OTP. Distance does not seem to have an impact on admissions. Populations from farther areas ( 4 km) seem to have more admissions compared to the estimations of populations in the area. Figure 11: Percentage of OTP admissions and percentage of population per distance in Km to the OTP. Meyu Muluke woreda, East Hararghe province, Ethiopia. February-June % 25% 20% 15% 10% 5% % OTP admissions % Population 0% > 15 Distance to the OTP (Km) 29

30 Qualitative data analysis The qualitative methods used included focus groups, semi-structured and structured interviews, cases studies and observations. Doing so revealed boosters and barriers. Interviews and focus groups were conducted in villages across the Meyu Muluke woreda. Questionnaire guides were adapted and oriented to facilitate the collection of data pertinent to program coverage and access. The investigation team also elaborated a list of terminology in the local languages (Annex 4) related to malnutrition and the RUTF. Qualitative data was triangulated by both method and source. All findings were indexed daily into the three-pane BBQ framework (complete BBQ can be found in Annex 5). Table 1 lists the sources and methods used during qualitative data collection. Questions ("Q") that appeared along stage one were analysed and resolved within days. Table 1. SQUEAC BBQ framework legend. Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013 Code Source Code Method 1. Community of Women 2. Community of Men 3. Community Leader/ Religious Leader 4. Mother/Caretaker SAM 5. Health Extension Workers (OTP), Health Workers (SC) 6. Health Development Army (HDA) A. Group Discussion B. Semi Structured Interview C. Case Study D. Observation E. Data Analysis F. Small area survey 7. Traditional Healer 8. Women Leaders 9. Staff of Health Center/ Head of HC 10. Woreda (woreda)moh 11. IMC project Staff 12. FDA Table 2 details the principal factors that either negatively or positively influenced program coverage and access during the qualitative data analysis; these are the main barriers and boosters. Table 2: Main program barriers and boosters after qualitative data analysis. Meyu Muluke woreda, East Hararghe province, Ethiopia. July Barriers Insecurity: Lack of OTP in some Kebele Cost of the time for staying in the SC as a family Rejection in OTP that is not their Kebele Not reporting some defaulters Refuse to go OTP because they assume they will need to go to SC ( Occupation, many children at home, looking for water for the family) Stock-out RUTF ( medicines) Boosters Awareness on Malnutrition Awareness on OTP & Appreciation IMC support (Logistic, material, training & supervision) Understanding of schedule of treatment Community network for OTP referral High level of engagement/ Commitment of MOH 30

31 Not acceptance of the HDA referral because not recognised Distance ( Pastoralist looking for water, HF far ) Lack of MUAC of many HDA ( no possibility of screening) nor training Mother sick Relating SAM with poverty; refuse to go to OPT of rich families Not aware that OTP is free, they go late Discontinuity of the IMC support Follow up of children by HEW (Referred to SC, absente) Good relation HDA,HEW, nurses (Weekly meeting) First Seeking Behaviour HP OTP site is close to the communities 31

32 4.2. STAGE 2 This stage confirms the location of areas of high and low coverage and the reasons for coverage failure identified in Stage 1 using small studies, small surveys, small-area surveys. The routine program, quantitative and qualitative data collected in stage one, when combined, helped identify areas within the intervention zone where coverage was likely to be either satisfactory or unsatisfactory. This information was used to formulate hypotheses about coverage that were tested. Small-area surveys methodology were used to test this hypotheses. It was difficult for the team to identify areas of low coverage because areas with insecurity were the ones identified as low coverage but they were not accessible. Analysis of number of admissions and discharged cured with MUAC < 110 mm was finally chosen to be the factors for identifying low coverage area. Table 3: Small-area survey selected villages for, Meyu Muluke woreda, East Hararghe province, Ethiopia. July 2013 Discharged Number of Other Villages OTP MUAC <110 admissions comments Low coverage areas Lami, Anano, B/qalla, B/guda, A/Hasan, Solom High coverage areas Hargaya, Mussa, Challa, Lucha CHIRRA 28.6% Low CHELLA 4.3% High Health Center. Staff very professional. The Lot Quality Assurance Sampling (LQAS) classification technique was used to analyze the data. The threshold value «p» that was 40% Low coverage: thirteen SAM cases were found (n=13); one case was not covered. Zero cases were found in the process of recovery. d = (13 x (40/100) =5.2 ~ 5. So 12 > 5» Not confirmation of hypothesis of low coverage area. - High coverage: seven SAM cases were found (n=7), two cases were found not covered. Four cases were found recovering. d = (7 x (40/100) =2.8 ~ 2. 5 > 2. Confirmation of hypothesis of high coverage area. The high coverage area was confirmed. The low coverage area was not confirmed. The investigation team decided to continue the investigation due to the difficulty of stabilizing hypothesis of low coverage apart from the insecurity areas. Like this the final idea was that coverage in all accessible areas could be homogenous and with high coverage. 12 The Centric Systematic Area Sampling (CSAS) coverage survey done in Babile woreda in November 24th to 5th of December 2011 by IMC was the guide to establish this threshold. 32