Evaluation team: Dr Thongdeuane Nanthanavone, Team Leader and Primary Author Dr Khampheng Phongluxa, Ms Amphone Chanthamith

Transcription

1

2 Final evaluation of Plan Laos Maternal, Newborn & Child Health & Nutrition Program And Mid-Term Evaluation of Water, Sanitation and Hygiene Program In Meung, Paktha and Pha Oudom Districts Bokeo Province, Lao PDR Evaluation team: Dr Thongdeuane Nanthanavone, Team Leader and Primary Author Dr Khampheng Phongluxa, Ms Amphone Chanthamith July-August 2015 i

3 Acknowledgement The consultants would like to acknowledge the support they received from the staff of Plan International Lao PDR, the Bokeo Team and District Team Leader of all three target district. The team expresses special appreciation to Hilda Winartasaputra, Dr Kalana Peiris, and Yuko Yoneda for their guidance throughout the evaluation process. Our special thanks also go to the National Managers of WASH and MNCH-N and their provincial counterparts without their supports the evaluation would not have completed successfully. The team also greatly appreciates the contributions from representatives of Plan implementation partners and other international organizations who generously gave their time to talk with the evaluation team. ii

4 Abbreviations ANC Antenatal care BEP Basic Education Program BCC Behavioral change communication CLTS Community Led Total Sanitation CWS Community Water System DEB District Education Board DHO District Health Office DHO District health office DRDO District Rural Development Office DRDPRO District Rural Development & Poverty Reduction Office DWTs District WASH teams ECED Early Childhood Education and Development EU European Union EPI Expended project immunization FGD Focus Group Discussion GFWS Gravity Fed Water System HC Health Centre HH Household/s HP Hygiene Promotion HW Hand Washing IEC Informational and Educational Communications INGOs International Non-Government Organizations KII Key informant interview LRC Lao Red Cross LWU Lao Women's Union LYU Lao Youth Union LSIS Lao Social Indicators Survey MCH Mother and child health M&E Monitoring and Evaluation MDG Millennium Development Goal MHV Model Health Village MNCH-N Maternal neonatal and child health and nutrition MoE Ministry of Education MoH Ministry of Health MoU Memorandum of Understanding MTR Mid Term Review NCA Norwegian Church Aid iii

5 NGO O&M OD ODF PADETC PCA PDR PED PES PHD PNC PRDPRO PW RWSS SBA S&H SIDA SLTS SO SoQ SSHT ToR ToT TT TWG UNICEF U5Y VEDC VHC VHV WASH WHO WRA WS WSP Non-Government Organization Operations and Maintenance Open Defecation Open Defecation Free Participatory Development Training Centre Participatory Community Assessment People's Democratic Republic Provincial Education Department Provincial Education Service Provincial Health Department Postnatal care Provincial Rural Development & Poverty Reduction Office Pregnant women Rural Water and Sanitation Strategy Skill birth attendant Sanitation and Hygiene Swedish International Development Agency School Led Total Sanitation Specific objective School of Quality School Sanitation and Hygiene Triggering Terms of Reference Trainers of Trainor Tetanus toxoid Technical Working Group United Nations Children's Fund Under five years Village Education Development Committee Village health committee Village health volunteer Water, Sanitation and Hygiene World Health Organization Women in reproductive age Water System Water and Sanitation Project (World Bank) iv

6 Contents Acknowledgement... ii Executive Summary... x 1. Introduction Country Context Plan International in Lao PDR The WASH and MNCH-N Programs The evaluation Purpose of the evaluation Desk review Primary Data Collection Data management and analysis Constrains and Limitations Results, achievement and progress Achievements of the WASH program Achievements and progress toward specific objective Achievements and progress toward specific objective Achievements and progress toward specific objective Achievements and progress toward specific objective Achievements of the MNCH-N program Achievements and outcome for specific objective 1 (SO1): Achievement and progress for specific objective 2: Achievements and progress toward specific objective Relevance Relevance of the WASH Program Relevance of the project design Water supply system rehabilitation and construction Sanitation component Hygiene promotion component Relevance of the MNCH-N Program Delivery mechanism Monitoring and reporting Focus on nutrition v

7 4 Effectiveness Effectiveness of the WASH program Hygiene promotion component CLTS Construction/rehabilitation of WASH facilities Participatory governance in WASH sector Working relationships with government partners Effectiveness of the MNCH-N Program Monitoring and Evaluation Impacts and synergy Impacts at the household level Impacts at the community level Impacts at the local government levels Synergy Sustainability Sustainability of WASH Sustainability of MNCH-N Scalability WASH MNCH-N Gender, Accountability and Inclusion Gender, accountability and inclusion in the WASH program Gender, accountability and inclusion in the MNCH-N program Conclusion WASH MNCH-N Recommendations ANNEXES Annex I: ToR ANNEX III: DOCUMENTS GATHERED AND REVIEWED ANNEX IV: DOCUMENTS GATHERED AND REVIEWED Annex V-A: HH Survey Annex V-B: Interview Guides ) KII guide for central level stakeholders vi

8 2) KII guide for local level stakeholders (PHO, DHO, DEO, HC) ) Interview: Local Authorities/CLTS/ VHC Village level ) FGD guides for Women, Men, and Children ) Observation checklist vii

9 List of Figures Figure 1: Number of children accessed health services during health events Figure 2: Number of women of reproductive age (WRA) and pregnant women (PW) accessed health services during health events Figure 3: A place to wash hands and hand washing with soap Figure 4: Knowledge about critical time to wash hands Figure 5: Knowledge about causes of diarrhea Figure 6: CLTS participation and access to sanitation by ethnic groups surveyed Figure 7: Sanitation coverage in target district against baseline Figure 8: ODF achievement in villages and schools Figure 9: the role of men and women in decision making about toilet construction Figure 10: The access to mother and child health services at baseline survey compared to the end line survey in Figure 11: The access to new born and child care at baseline survey compared to the end line survey in Figure 12: Wasting (Weight for Height/length) by district Figure 14: Stunting in three target districts Figure 15: Stunting by age in month and target district Figure 17: Persons responsible for fetching water Figure 18: Fee payment by Plan and none-plan supported CWSs Figure 19: Fee payment by Plan and none-plan supported CWSs viii

10 List of Tables Table 1: Timeline of key events for both programs... 9 Table 2: Sampling frame for the household survey and FGDs Table 3: The targets and the reported achievements of the objective Table 4: The targets and the reported achievements for the CLTS intervention component (source: annual narrative reports ) Table 5: The targets and the reported achievements for the WASH infrastructure construction Table 6: The targets and the reported achievements for the WASH governance Table 7: Achievements of establishing of MHV in three target districts Table 8 Number of annual and quarterly meetings and number of staff attended Table 9 Training courses organized between 2012 and 2014 and number of participants attended the training course Table 10: Summary outreach activities and health events held in communities between 2012 and 2014 and number of beneficiaries attended Table 11: Summary of direct beneficiaries accessed to each type of health services in communities between 2012 and Table 12: Summary of estimated water and sanitation conditions in the 4 target districts Table 13: Number of people with increase knowledge of hygiene practices Table 14: Number of people participated in the CLTS sessions Table 15: CLTS participation and construction of the toilets Table 16: Status of delivery at baseline and end-line survey ix

11 Executive Summary Background Lao PDR has experienced an average economic growth of 8% and has moved up the ladder to a low-middle income country status in However, as experienced in many emerging economies, inequality has been going up and the distribution of the benefits from the growth has not been even particularly between urban, rural and remote sectors. Data from the WHO/UNICEF Joint Monitoring Program 1 showed that at the coverage of 76 % (water) and 74% (sanitation), Lao PDR had met the MDG targets for both. However, that still left almost a quarter of the population (24 %) without access to improved sources of drinking water and almost one third (29 %) without access to improved sanitation. Links between WASH and under-nutrition are well-established. The World Health Organization (WHO) estimates that 50% of malnutrition is associated with repeated diarrhoea or intestinal worm infections as a result of unsafe water, inadequate sanitation or insufficient hygiene 2. Commonly caused by consumption of water or food contaminated by human and animal faeces, diarrhoea is directly related to poor sanitation and hygiene conditions. Diarrhoea is a leading cause of death in children under-five globally and its constant presence in low-income settings contributes significantly to under-nutrition. A lack of access to clean safe water within close proximity to home has many indirect effects on nutrition. People are often left with no choice but to drink unsafe water from unprotected sources. The time spent collecting water, bringing it home or suffering from water-related illnesses prevents young people from getting an education, which has a significant impact on their health, wellbeing and economic status. In Bokeo, sanitation reflects national trends of low coverage. It is particularly so for the three target districts of Plan. Sanitation coverage rates are below national target rates, especially when considering actual usage. Open defecation rates are still high, despite the provision of toilet materials to village households in the past. Maternal Mortality Rate, Perinatal Mortality Rate (IMR & <5 MR) and stunting levels are high. From a health sector point of view, service coverage is low; service quality is poor and cultural and physical barriers are many and consequently the service usage is low. Water Sanitation and Hygiene (WASH) and Maternal, Neonatal, Child Health and Nutrition (MNCH-N) Programs The first phase of the WASH program in Bokeo began in The MNCH-N started 2011 when the second phase of WASH started. Both have the Memoranda of Understanding signed with Provincial Health Department (PHD). The largest grant for the WASH program runs until June 2016, so this evaluation serves as a mid-term evaluation for WASH. The project has initially focused on 3 districts in Bokeo and was expanded into Oudomxay and Saravane provinces in The main grant for the MNCH-N program has been implemented only in Bokeo, since the inception of the MNCHN programme. This project has come to an end in June Following the inception of the MNCHN programme in 2011/12, a program review was undertaken in 2013 to ensure Plan s MNCH work is aligned with MOH priorities and global 1 Progress on Sanitation and Drinking Water, UNICEF & WHO (Joint Monitoring Programme), 2015 Updates 2 World Health Organisation (2008c) Safer water, better health: Costs, benefits and sustainability of interventions to protect and promote health. x

12 best practices. This resulted in a revised approach for MNCHN. This evaluation will serve as the end of project evaluation for this project. The overall objective of the WASH project is stated as Improved health conditions resulting from changed and sustained WASH behaviours among children, women, and ethnic and other vulnerable groups in 170 selected villages in Pha Oudom, Mueng, and Paktha districts of Bokeo and in Houn district in Oudomxay province by 2016, thereby contributing to Lao PDR s MDGs and development goals for It has four inter-related outcome objectives: 1: Improved and maintained good personal and household hygiene, hand washing with soap at critical times, and consuming safe drinking water among community members, particularly among women and children in selected target communities and schools; 2: Achieved and maintained ODF status in 80% of the selected target villages and schools; 3: Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages; and 4: Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level. At the district level, Plan established a technical working team called District WASH Teams (DWT). The DWTs are multi-sectoral teams comprised of technical staff from District Education and Sports Bureau (DESB) and District Health Office (DHO) as well as members of Lao Women Union (LWU) and Lao Youth Union (LYU). Four closely linked set of activities were implemented to achieve the above objectives. These are Community-led Total Sanitation (CLTS) in the target villages which include pre-triggering meeting with community, triggering sessions with communities and post-triggering follow up session by the District Wash Teams (DWT). A similar set of CLTS activities were conducted in schools called SLTS. WASH facilities (toilets, hand washing stations and water supply) were provided in schools and Water Supply to some communities to further promote improved sanitation and hygiene practices. These activities were re-enforced by Behaviour Change and Communication (BCC) including WASH related awareness raising events. The Overall Objective of the MNCHN project is to improve the health and nutrition status of the population of three Bokeo (Pha Oudom, Mueng, and Paktha) Districts in Lao PDR, through an integrated primary health care approach that emphasizes the health of women and children. The program has three strategic objectives (SOs): 1. To contribute to the development of a sustainable Primary Health Care system in Bokeo Province; 2. To promote Universal Access to Quality Health Care for the families of three Bokeo Districts; and 3. To engage Bokeo Communities for health by seeking their advice, and encouraging local reflection, prevention and promotion activities A number of activities were implemented to achieve these objectives. The first set of activities designed to develop a sustainable primary health care system included strengthening the health management information system, strengthening district, health centre and outreach xi

13 management and improving the quantity and quality of human resource for health and nutrition. The programme also promoted universal access to quality health care through health day, mobile clinics, nutrition education and food demonstration, and children day. The programme also promoted health care seeking behaviour by encouraging care at the household level, improving awareness of community about the rights, needs and problems related to pregnant women, new born and children, and improving health service providers to make them user-friendly. The evaluation The evaluation employed a mixed method combining quantitative and qualitative methods. The evaluation followed the assessment criteria set in the ToR including: relevance, effectiveness, impact and synergy, sustainability, scalability, gender, accountability and inclusion (in relation to Plan s Child Centred Community Development standards). Sampling was purposive to capture the changes in the most remote villages with higher percentage of poor Households and people representing ethnic (minority) groups. The method employed included: Focus Group Discussions (FGD), Key Informant Interviews (KII), Household Survey (HHS) and site observation. Findings 1) Relevance The WASH and MNCH programs both were relevant in terms of responding to the needs of the communities. In the baseline, the target communities had recorded a low rate of sanitation coverage, limited access to safe drinking water and poor hygiene practices. On one hand, the WASH program employed the CLTS approach to promote behavioural change in sanitation and hygiene practices at school and community levels. Adoption of the CLTS approach has been supported by the government through its recognition of the approach in National Plan of Action for Rural Water Supply and Sanitation 2012 and endorsement through the WASH Technical Working Group. The school hygiene and sanitation component is well aligned with the education sector strategy regarding improving hygiene practices amongst children. On the other hand, the MNCH-N Program in Bokeo is aligned with government policy and relevant to the current needs in the province. The project design was consistent with the need of the Lao government in providing the universal access to health care services particularly for remote rural population with the aim to reduce the malnutrition and child and maternal mortality rates. It is evident that the MNCH-N Program in Bokeo had been designed to implement through the existing health care system of the province including supporting to strengthen the capacity of PHD, District Health Office (DHO) and Health Centre (HC) staff. The program had made considerable effort to make the program remain relevant and consistent with the government policy. For example, government institutions like Nutrition Centre, MCH Centre, and EPI were invited by Plan to be core lecturers so that it reflected the common understanding concerning to MNCH-N lessons and ensured the implementation goes in the same direction. It had achieved the contextual harmonization of working groups between project staffs and government partners. This continued engagement with the government counterpart also ensured the Program remained relevant and coherent with the government plan and approach. xii

14 2) Effectiveness Overall, the evaluation found that the project s hygiene promotion activities were effective in raising awareness and increasing knowledge about improved hygiene practices. The results varied in terms of behavioral change promotion. Students both boy s and girl s groups reported that they drink water treated and stored properly. However, we also observed that some children drank water directly from the taps (public water points) suggesting that this knowledge has not necessarily been turned into practice. Improved hygiene practices at HH level is reported to have been adopted by villagers in target communities. FGD respondents reported that they did not have a concept of washing hands with soap before the project started. Now they learned that they should wash hands with soap at critical times (such as after defecation, after cleaning children s bottoms, before eating, before feeding children and before preparing food). In addition, the HHS shows that the hygiene and sanitation in villages had been improved: so far achieved 78% of the project target in terms of number of Open Defaecation Free (ODF) villages, 34.5% of mothers had better knowledge about how to dispose children s faeces properly. More people knew about three critical times to wash their hands (Meung 23%, Paktha 12% and Pha Oudom 35%) This evaluation also confirmed that the CLTS approach was effective in promoting private investment in household (HH) sanitation. The evaluation proved that community members who participated in the triggering sessions tended to construct new toilets than those who did not. The evaluation also found that there were some other factors influencing household s investment in the toilet construction. These include subsidies from the government or a history of subsidies by other projects, availability of employment which increases the financial capacity of the villagers and access to the sanitation markets. Construction/rehabilitation of water supply systems has contributed to the effectiveness of Hygiene Promotion (HP) in schools as the facilities (where functional) provided beneficiaries in schools with the means to adopt and practice the recommended sanitation and hygiene practices. Gender equality was also promoted at HH level. However, the role of women making decision about toilet construction remained minimal. Overall, the MNCH-N program had successfully achieved its objectives. As the HHS indicates, access to Ante Natal Care (ANC) visit 4 times, Post Natal Care (PNC) visit within 7 days of delivery, PNC visit within 6 weeks and PNC visit 3 times had increased over time and was particularly so comparing to the baseline survey. In addition, mothers had better knowledge on four danger signs to take a child to health facilities (7.7% in this survey compared to 3.4% baseline). This is due to the parenting orientation as well as the MNCH-N that synergistically delivered mother and child health messages. Knowledge of mothers on danger signs of pregnancy and delivery had been improved. The survey shows that more mothers were aware of at least two or 3 critical signs at postnatal period that need care at health facilities (39.6% and 18.9%, respectively). Concerning the nutritional status of children < 2 years, this evaluation observes that the program has improved the nutritional status of the target children particularly for wasting (weight for height/length) among children < 2 years. As shown in this HHS, the percentage of severe and moderate wasting has decreased from 6.5% and 13.7% in baseline survey to 4.3% and 6.5% in this HHS, respectively. In addition, the MNCH-N program activities succeeded to encourage mothers to give babies the initial feed of breast milk within one hour of birth xiii

15 (74.8%), and to give complementary food beside the breastfeeding to children 6-9 months (92.0%). However, there are some areas where the program results were less successful. Regarding delivery, the program had not achieved the set target to encourage mothers to give birth at health facilities with Skilled Birth Attendant s (SBA) assistance. The KII respondents reported that it is hard to change this behaviour within a short time frame because this culture is deeply rooted in most Lao mothers. The HHS indicates that the program had not achieved the target of reducing stunting (height for age) among children < 2 years. The percentage of severe and moderate stunting in this HHS was higher than the baseline (22.6% and 41.6% vs 17.1% and 38.2%, respectively). 3) Impacts and synergy Plan-supported Community Water Systems (CWS), and those supported by other agencies, had reduced women s workload. As women are responsible for fetching water, CWS reduced time and energy women have to spend on collecting water. FGD participants (women groups) reported that they were happy that they did not have to carry two buckets of water home sometimes as far as 10 km for household use and drinking. Now they could save time and energy from this reduced workload. They reported to be freer now to watch TVs, learn some cooking lessons from there, learn how to properly care their children etc. Furthermore, more villagers were able to connect water to their dwelling (PVC pipe or plastic pipe) which freed up women from water collection duty. All FGD participants reported that positive behavioral changes concerning using toilet, washing hands with soap, drinking safe water and keeping environment clean had been taking place after the Plan s interventions. Almost all of the FGD participants were aware of the importance of using toilets, washing hands with soap at critical times, drinking safe water and keeping surrounding environment clean. The positive and negative changes were found at the household level after the introduction of MNCH-N programs activities. For example, percentage of mothers who had ANC visit 4 times had increased by about 20%. Women of Reproductive Age (WRA) who received vaccination against Tetanus increased by about 25%. However, there remain some challenges regarding mother and child health. Pregnant Women (PW) still worked hard in the field as elder people taught them that working hard would help make delivery easier. In addition, PW did not have access to diverse and quality food. There was no special attention being given to diets of PW. Results of HHS indicates that the knowledge on danger signs of PW and PNC, although still low, has improved (knowing 2 critical signs of PNC 39.6%, knowing 3 critical signs of PNC 18.0% and knowing critical sign of labor 18.0%). Program has shown some positive changes in terms of gender equality. It was reported that now more women become committee members and even sitting at leadership positions. Children s rights and participation were recognized at HH and community levels i.e. their participation in the school activities and planning process in the communities (initiated by Plan). These changes had never taken place before the program started. The program has not yet involved people with disability (PWD) and this survey found some family members had taken care of the people with hearing impairment. The community members reported that before, children were frequently sick of diarrhea and some children died. The village authorities interviewed reported that over the last three years xiv

16 there was no case of under-five mortality in the villages surveyed. The integration of WASH and MNCH-N programs has given a positive impact not only on the acceleration of the development of Model Healthy Villages but also addressing malnutrition among children < 2 years particularly wasting. From the field observation, the team found that very few sanitation facilities were constructed in villages where there was no WASH program present. At the same time, more women and men reported seasonal diarrhoea outbreaks in villages where there was only MNCH-N program presented. 4) Sustainability Sustainability of hygiene promotion in schools rested on the integration of hygiene promotion activities into school curriculum and the hygiene promotion activities led by teachers and model/volunteer students. Sustainability of the WASH facilities depended largely on the ownership of the committees. It was reported that the school administration fund was the only source of money to be spent on maintenance of school WASH facilities. The committee reported that minor maintenance could be done by the CLTS committee members. According to education officials interviewed, the schools that meet the government SoQ criteria would receive encouragement fund (1000kip per student per year). However, none of the schools that had met the criteria had received the money yet. The Institutional structure developed under the project related to school WASH is the establishment of DWTs, Village Education Development Committees (VEDC), CLTS committees and involvement of village chiefs and village cluster heads. Over the project period, these entities were actively involved in the school WASH activities. The key to sustainability of CLTS are continuous follow-up among households to ensure maintenance of toilets and sustained use. Plan is striving to ensure local ownership and community-led processes in relation to WASH. Local governance structures such as VEDCs and CLTS committees were established to empower community participation. However, across the three districts visited, officials both Plan and government counterparts, reported that community ownership remained an issue of concern. At community level, even village authorities, still believed that WASH facilities belong to Plan. Villagers were under the impression the project is owned by the district or provincial departments. Fee collection by the water committees is a crucial source of funding for maintenance. The results from HHS show a low rate of fee payment contributing to water fund at the community level with only 37% paying the fee regularly. This result is vastly different from the rate at which it was reported in the 2015 annual report (80%). Sampled villages for the HHS being the remotest with higher percentages of poor HHs could explain this difference, which also shows that alternatives should be sought for poorer and more remote communities when it comes to sustainability. When the Plan s CWS are compared with other villages supported by other organisations, the result is not significantly different. The success of fee collection or maintenance fund management is also dependent on the sustainability of the CLTS committees themselves. Sustainability of CLTS committees at village level remains a great challenge. One of the reasons is that at the moment they are not functioning as they supposed to be. From interviews with CLTS committee members, the team found that the CLTS committees stand-alone without any links to the other village development committees such as the Village Health Committee (VHC) mandated by the government to carry-out health activities, members have limited skills in encouraging other xv

17 villagers to improve sanitation and in many villages, the committee members themselves had not built toilet yet. Another reason is the lack of incentive/reward for the effort for collective gain. This together with the fact that they are not fully volunteered in becoming the committee members can lead to unsustainability of the initiative itself. It is unlikely that DWTs will continue to function after the financial support from Plan has ceased. Therefore, CLTS activities must remain and be carried out by VHC members (in this case CLTS at community level need to be integrated into VHC). It is likely that working with existing committees will ensure sustainability. Sustainability of MNCH-N activities in community depends on the ownership of health care providers including village authorities and actors like Village Health Volunteer (VHV), VHC and village facilitators in organizing an outreach activities and health events in communities. It is likely that the activities initiated by the program will be sustainable as they have been aligned with the government policy and constantly responding to the needs of the community. This is well evident throughout the program implementation. Stakeholders at the central level acknowledged that Plan has contributed significantly to improvement of the mother, newborn and child health and nutrition in Bokeo. Plan was well regarded as contributing to supporting these MNCH-N projects in remote provinces of Lao PDR which have been facing funding shortages. Sustainability of outreach activities and health events in communities, local stakeholders (KII) shared his view that if the program is ended the outreach activities would be continued however the coverage and completeness would be less, and vaccination and health promotion is unlikely due to the government has no budget. One KII respondent stated that integrated mobile clinics might not be possible. Now health staffs were able to use computer for routine work and conduct health events in village and knew how to do microplanning. Program had supported the renovation of health centres including supporting medical equipment and water supply and sanitation facilities in health facilities, logistics support to 3 target districts and official equipment and materials were supported by the program. Annual and quarterly meetings with the PHD and key government counterparts ensured opportunities for reflections, planning, and adjusting the project plan if needed. In terms of accountability upward, reporting to the donor and government counterparts, the project has done considerably well. All the necessary information was reported back to the donors. Plan promoted ownership amongst both provincial and district government counterparts over the implementation of programs activities in communities. PHD and DHO staff took lead in monitoring and supporting HCs. The implementation of some activities, however, were delayed and not implemented as according to plan due to staff turnover within Plan and the counterparts alike. As a result, some key information such as population figures for the target group was not collected. 5) Scalability CLTS as an approach should be scalable. The team found that the approach not only increase toilet coverage but also made progress on stopping Open Defaecation. A working approach of DWTs should be rearranged. For sustainability purpose, the team should only work together at the triggering stage only; an individual team member can take on a follow up role xvi

18 over a certain number of target villages. We found that the current form of DWTs would not be sustainable. Construction and rehabilitation of CWSs should be scaled up. It is strongly evident that water supply and availability of water all year round is the critical factor for HHs to get access to improved sanitation practices including building and using toilets and washing hands with soap. Another key intervention that should be scaled up is the application of Gender and WASH Monitoring Tools (GWMT) beyond mere monitoring to supporting communities to tackle the identified gender inequities. The team found that WASH improvements reduce women workload and, to address gender inequality it needs to be designed and monitored well. In addition, the KII respondents emphasized that a proper monitoring system has to be established to follow up and support communities because communities are very poor and people have low, self-awareness. Regarding the scaling up of activities some challenges need to be carefully considered: lack of staff particularly at health centre and DHO, limited capacity to perform required tasks and staff who are used to low usage rates may consider an increasing usage rate over whelming. The establishment of many new committees in villages should be avoided. 6) Gender and social inclusion It was found that CLTS sessions at the village level remained dominated by the heads of the households and they are usually men. The government officials attitudes toward different roles and responsibilities of men and women have changed since they had participated in the training about gender and social inclusion organized by Plan. Although no quotas have been set for women for trainings, DWTs, CLTS members, etc. women are generally promoted to take part into the committees and trainings and given equal opportunity to participate in Program activities. For example, 20 provincial and district health officials (including 10 female) had been trained on CLTS in Bokeo. Program staff keeps record on training, workshop as well as seminar participants disaggregated by gender. Other disaggregation (by ethnic background etc.) is not performed. The effort to reach more marginalized and people living in the most remote areas intensified during FY15. The project s target villages included more remote communities with more difficult access. With these challenges Plan introduced an interns system where it drew young people from target communities to get involved. There is little evident to show that disability has been integrated in the project designs. An interview with Plan staff confirmed that they had participated in the disability classification workshop which helped them to recognize more comprehensively about types of disabilities. In terms of physical facilities of WASH and health facility WASH, we observed that technical designs of water points have not been disability sensitive. 7) Conclusions and Recommendations The recommendations for the future direction of the program are provided on short-term and long-term basis. xvii

19 Short-term recommendations 1. Despite impressive achievements in many respects of the program, the nutrition status of the target children under 2 years old has yet to achieve the target due to poor feeding practices and poor sanitation and hygiene conditions. As the national trend toward refocusing on nutrition is rising, it is important that Plan joins the government and Development Partners (DPs) by using the existing platforms such as SUN to scale up WASH in Nutrition approach. 2. WASH and MNCH-N programs should be integrated as the outcome from WASH program can complement the outcome of other program. Synergized outcomes could provide positive impact to the nutrition status of children under 2 years and mother neonatal and child health and nutrition in community. 3. At the moment, there is no link between WASH promotion at village level and the Health Center staff. With a smaller DWT some WASH responsibilities should be delegated to HCs because they are close to communities and have the overall responsibility of the health status of people in the catchment area. 4. WASH and MNCH-N can jointly operate (convergent implementation). At the community level with CLTS committees, water committees, VHC integrated into one committee but keeping members with mechanical skills for toilet construction and water supply system maintenance. 5. WASH, particularly water supply should provide water beyond drinking and household use to cover animals raising and gardening. By doing this WASH can contribute more directly to improvement of nutrition outcome in the communities because we know that part of the root causes of malnutrition are the lack of access to food. Animal raising and gardening can increase food availability thus access to food. 6. Food hygiene can be integrated into intervention. Hygienic food preparation and storage should be integrated into the intervention. 7. Some WASH facilities in schools and communities started to break. It is important to urgently address this problem. To do this, the project has to change its monitoring and follow up arrangement. Follow up support beyond one year is needed to help the party responsible in the village level build their competence and experience. 8. Rather than working from Plan office, provincial managers should take the office space provided by PHD. This will help foster better coordination and communication. 9. The DWTs are unlikely to be sustainable and remain effective beyond the project life. Number of DWT members should be reduced to two or three as the bigger the team the more unlikely it will be sustainable and effective. This is to redirect more resources into community level. Plan should rely on the village level. It is important that Plan start to build a strong and competent WASH team at the community level, ensuring the each community has a fee collection system that can cover the minor maintenance cost. Sustainability of the results of the interventions relies on the organisation at the community level with only periodic support from the district level. 10. It is important to make sure a strong governance system at the community level is in place. This includes transparency of the water fund management, strong and competence technical person who can deal with practical repair and maintenance of the facilities. Long-term recommendations There was consensus among key stakeholders that CLTS works and could lead to effective use and behavioral changes amongst target population. It is recommended that Plan: 11. Promotes cohesion within the WASH sector concerning adoption of CLTS that is still lacking. Plan should work with MOH and through WASH TWG to advocate for wider adoption of the approach. At the moment, there is an ineffective coordination or communication between central and local government counterparts, such as variation of their view on CLTS approach. xviii

20 12. Plan should also continue to increase efforts to get other stakeholders on board in terms of CLTS adoption. This will include clarifying with key stakeholders that CLTS works but takes time to get the results. 13. School WASH is clearly inextricably linked with the rights of the child and enhanced child survival and must therefore remain a key priority for Plan 14. Convergent planning and implementation for a more integrated, comprehensive WASH and MNCH-N must be strengthened to build the potential synergies. 15. Maintenance of WASH facility in schools should be the responsibility of the VEDC because in the foreseeable future there will not be any improvement in the budget situation of the school. 16. Engage more actively with the sanitation material suppliers in terms of outreach activities and advertising of their products to the local communities. 17. Apart from addressing the communication (intern)/engagement strategies (language) adopted by Plan, an approach where specific interventions are amenable to change at community level by different ethnic groups may need to be considered. For example, the prescription of solutions by District Authorities model should be changed to solution exploration by community model. Furthermore, slowest groups might need more engagement and more encouragement or different approach/strategies altogether. 18. Report on program of Plan should be distributed during the meeting of technical working groups or health research forums. To make sure that the concerned institutions at the central do miss out on the progress of the program. 19. It would be effective in implementing if DHO and HC meet each other and discuss about the role and function of organization to lead the health event in villages. DHO should work closely with health providers and encourage the assigned staffs to perform their tasks as expected and with high responsibility. 20. PHD should report about the Plan program implementation to MOH and national health forums. 21. The representative from the Central Ministry who attended the annual meeting should report to its Director in a written manner. 22. Generally speaking, malnutrition has not seen significant improvement, PHD and DHO together with Plan should coordinate with local agriculture and forestry for the promotion of home gardening, protecting and nurturing natural sources as good food source for community. At the same time some knowledge on the important of eating variety of food and more food groups should be explained carefully and in ethnic language. 23. The delivery at health facilities rate remains low, due to cultural barriers among ethnic women, such as they were too shameful to let other people see their PHD and DHO have to be sure about gender of health provider at health centre, at least one women should be presented at HC for delivery of health service particularly MNCH-N services. PHD should discuss with Plan or other international organization for more SBA development to ensure that delivery attended by SBA and PHD should consider about recruiting the SBA produced by Plan into HC level. xix

21 1. Introduction 1.1 Country Context Lao PDR is a land-locked country in South-East Asia. It is a very diverse country with 49 officially recognised ethnic groups. Ethnic groups, which make up almost half of the population, mainly live in highland areas. The government of Laos (GoL) with technical and financial support from Development Partners (DPs) has been addressing the infrastructure and societal gaps between the lowlands and highlands. However, the differences are still significant, especially in terms of access to basic public services. Regardless of the progress made by the economic sector, social sectors development has been lagging behind. In the last decade, the country has enjoyed an average Gross Domestic Products (GDP) growth of 8%, and in 2012, became a low-middle income country. Yet, inequality is increasing and rural and remote ethnic communities are not benefitting from the economic growth. In other words, the significant progress in the social development has not been fully going hand-inhand with the reported economic growth. At the score of 0.569, the UN 2013 Human Development Report positions Lao PDR as the 139 th out of 187 countries and territories in the World 3. Significant progress has been made in the provision of water supply and sanitation services across the country. Millennium Development Goal (MDG) No. 7 commits countries to halve by 2015 the proportion of people without sustainable access to safe drinking water and basic sanitation. Data from the WHO/UNICEF Joint Monitoring Program 4 showed that at the coverage of 76% (water) and 74% (sanitation), Lao PDR had met the MDG targets for both. However, that still left almost a quarter of the population (24%) without access to improved sources of drinking water and almost one third (29%) without access to improved sanitation. Moreover, disparities between urban and rural areas have also persisted. In rural areas, where the majority of the poor live, access remained well below that of the urban areas: 69% and 58% for water and sanitation respectively. Only 38.9% of all schools currently afforded access to WASH services including for students and teachers 5. Given the recent global evidence links open defecation and inadequate disposal of human faeces with stunting, it is highly concerned that in Lao PDR, open defecation remains widespread and a challenge for mother and child health. Furthermore, the Lao Social Indicator Survey (LSIS) reported a National Maternal Mortality Ratio (MMR) is 357 per 100,000 live births; an Under-five Mortality (U5M) at 73 per 1,000 live births; Infant Mortality Rate (IMR) at 68 per 1,000 live births with Perinatal Deaths (PNMR) accounting from 32 of these 6. The MDG target for maternal mortality is 260 per 100,000; for U5M is 70 per 1,000 live births; for IMR is 45 per 1,000 live births and for stunting is 34% in U5. Lao PDR continues to have very high chronic malnutrition rates: the stunting rate among children U5 is 44% nationally or nearly every second child under the age of five in Lao PDR is stunted. These rates are even worse among ethnic groups living in remote areas. Every fifth rural child is severely stunted. A nutrition survey in a remote, mountainous community in 3 United Nation Human Development Report 2014: sustaining human progress 4 Progress on Sanitation and Drinking Water, UNICEF & WHO (Joint Monitoring Programme), 2015 Updates 5 Lao Ministry of Education s Education Management Information System (EMIS) 2010). 6 Lao Social Indicator Survey

22 Sekong Province found a stunting rate of 67 %. 7 Micronutrient deficiencies affect large parts of the population, with over 40 % of children under five and 63 % of children under two suffering from anaemia, and almost 45 % of children under five and 23 % of women between 12 and 49 years of age affected by vitamin deficiency. On the current trends Lao PDR is unlikely to reach the mother and child health targets even though the under-five mortality rate is very close to the MDG target (from 73 to 70). In sum, urban-rural disparities, gender inequality, limited coverage of services/supply, unhealthy behaviours have been the root causes of poor health indicators. In recognition of these challenges, in 2013, the GoL and the Ministry of Health (MoH) launched the National Action Plan for Rural Water Supply and Sanitation and Operational Guidelines for Scaling up Rural Sanitation in Lao PDR. These guidelines were developed to provide a systematic framework for the planning, implementation, monitoring and evaluation of rural sanitation Programs under the umbrella of the National Plan of Action. This includes the roles and responsibilities of relevant state agencies at all levels. The intention is to ensure a coordinated approach nationwide, based on common national goals and priorities. The guidelines are founded on five guiding principles: 1) equity to ensure that all people living in remote areas have access to improved WASH services; 2) sustainability of services and behaviour changes; 3) sanitation demand creation and strengthening supply of sanitation goods and services; 4) shared responsibilities between stakeholders (households, government, public sector); and 5) learning and innovation. In the area of maternal, new born and child health and nutrition, the MoH has spent a great effort to reduce maternal and child death and free from malnutrition, food insecurity and poverty. The strategy and planning framework for the integrated package of maternal neonatal and child health services and national nutrition strategy and plan of action were developed as the best platform for MNCH-N implementation at all levels within the country. Currently the free delivery policy has been issued and implemented in all districts of the country particularly in remote districts 10. Furthermore, there has been an increased focus on improved nutrition in women and children, from the highest level of the government. A Prime Ministerial Decree was enacted in mid-2013 establishing for the first time an inter-ministerial Nutrition Committee, and Secretariat. In late 2013, a UN Task Force (IFAD, UNICEF and WFP) developed a new Food and Nutrition Security Action Plan (FSNAP- December 2013), based on the convergence model of multi-sectoral planning and monitoring, and sectoral implementation, across four sectors (Education, Health & WASH and Agriculture). Phase one of this plan prioritises Luang Namtha, Oudomxay and Saravane, with phase two adding Phongsaly, Xieng Khouang, Houaphan and Sekong. It is important to note that Bokeo is outside these priority provinces though the planned expansion to Oudomxay and Saravane are aligned with the multi-sectoral priority 7 CARE Nutrition Survey in Dak Cheung 8 Strategy and Planning Framework for the integrated Package of maternal Neonatal and Child Health Services National Nutrition Strategy and Plan of Action Decree on subsidy delivery and treatment children under five years of age, No 273 and dated The policy is a form of results-based financing (RBF) that will remove user fees and charges for medicines, provide beneficiaries with small incentive payments, and reimburse health facilities for provision of MCHrelated care. The free MCH policy builds on the relatively positive experience of several smaller-scale donorfinanced pilots that have implemented similar interventions in selected regions of Lao PDR over the past few years. 2

23 targeted provinces. At the time of this evaluation the inter-ministerial National Nutrition Committee, supported by the MOH and Food and Agriculture Organisation (FAO) are revising the government s National Nutrition Policy and Plan of Action, and adopting the convergence model and 22 priority nutrition actions identified in the FNSAP. Approval of the new government policy is expected in the latter half of In light of the challenges and policy context outlined above Plan International Lao PDR (Plan) has focused on the WASH and MNCH-N interventions. In carrying out its interventions, Plan also take into account gender inequality including deeply embedded cultural norms and practices, such as early marriage, women s workload, lack of their voice in household decision making as well as lack of access to information, resources and services on health, nutrition and WASH. 1.2 Plan International in Lao PDR In response to the emerging policy context as well as the community needs, Plan collaborated closely with the GoL, primarily Nam Saat Centre under the Department of Hygiene and Health Promotion in the MoH. It also collaborates with other international development agencies working in sanitation and hygiene promotion - particularly the World Bank-WSP, SNV and UNICEF. This would help to foster an adoption of a common approach i.e. Community led total sanitation (CLTS) to be scaled up within the framework of the National Plan of Action. It is also important to note that Plan intends to address gender inequality, limited coverage of WASH in remote areas and unhealthy/unhygienic behaviours that underlies poor health/nutritional outcome for women and children. It is evident to some extent that some key principles of gender equality were integrated into both WASH and MNCH-N programs. The evaluation assessed gender inequality including deeply embedded cultural norms and practices, such as early marriage, women s workload, lack of their voice in household decision making as well as lack of access to information, resources and services on health, nutrition and WASH. These are very important preconditions that will be incorporated in the analysis for this evaluation. Plan works with the Government of Lao PDR and the Development Partners (DP) to address the issues faced by marginalized children, especially girls, through participation, partnerships, advocacy and high quality Programming. The 2 nd (current) Country Strategic Plan (CSP) has five key Country Programs (CPs) This evaluation focuses on two of the CPs. Some background to the two programs is presented in the next section. 5 Country Programmes of Plan International in Lao PDR 1.Early Childhood Care and Development CP(ECCD) 2.Basic Education CP (BEP) & Child Protection CP (CRCP) 4.Water Sanitation and Hygiene CP (WASH) 5.Maternal, Neonatal and Child Health and Nutrition CP (MNCH-N) These two Countries Programmes are expected to directly contribute to the following key results of the CSP 1. 78,000 children 11 and their families demonstrate improved sanitation and hygiene practices in 400 open defecation free villages. 2. Less than 20% of boys and girls under 5 years of age are stunted in Plan target districts 12 Source: 2 nd Country Strategic Plan , Plan International in Lao PDR 11 Approximately 50% of the total population of children U18 in 9 Plan target districts of Pha Oudom, Paktha and Meung (Bokeo), Pak Beng, Houn and Nga (Oudomxay) and Saravane, Ta-Oi and Samoui (Saravane) 12 Stunting rates for children U2 in 2013 in Bokeo are: Pha Oudom: 48.8%, Meung 37.7%, Paktha 24.8%. Stunting rates per district are not available for Oudomxay and Saravane, However, the stunting rates per Province for children U5 are: Oudomxay 54.9%, Saravane 54.4% 3

24 1.3 The WASH and MNCH-N Programs Both Programs targeted the mountainous districts of Meung, Paktha and Pha Oudom of Bokeo which are amongst the poorest districts in the country. The total population is 73,500 and characterized by high ethnic diversity. Major ethnic groups include the Khmu, Lamed, Hmong, Lue, Akha, Muser, Tai Dam and the majority Lao. These make 80 % of Bokeo s population with the Khmu being the largest 13. There are 146 villages within the target districts of which 49% are classified by GoL as being poor. Concerns over cost of transport and treatment in addition to service quality were widely reported as factors limiting household access to formal health care services and in particular SBAs. In 2010, the WASH Program began its piloting of CLTS in three communities before expanding its activities to 35 villages and schools in Pha Oudom, Paktha and Mueng in Initially, WASH activities were meant to provide better sanitation, hygiene conditions and knowledge for children in target schools. However, to ensure greater impact, expansion to school-hosted communities was encouraged and implemented under the MoU between Plan, Provincial Education Department and Ministry of Education and Sports (MoES). For WASH, the core funding of the program comes from Australian Aid s ANCP and Plan Australia. In 2012, MNCH-N Program began to implement its activities across the three districts. Since then, the main MNCH-N grant under the MNCH-N Program has been implemented only in Bokeo. This project came to an end in June Following its inception in 2012, the MNCH- N Program review was undertaken in 2013 to ensure its alignment with MOH priorities and global best practices. The current evaluation will serve as an end of project evaluation for this project. However, the Memorandum of Understanding (MoU) which Plan has with the PHD for the Program outlasts this project and runs until December Plan intends to continue to engage with MNCH-N with a renewed focus on nutrition, in a way that is integrated with WASH. Program design and components Figure 1: Map of Bokeo Province with relative positions of the three Districts where Plan works The WASH project design was developed in a workshop in 2012 involving government stakeholders. It was agreed that the multi-sectoral approach to implementation was effective and would be adopted. Accordingly, the new MoU between Plan and Provincial Health Department (PHD) makes specific reference to the multi-sectoral partnerships and broadly defines the roles and responsibilities of each of the four sectors involved, i.e. DHO, DESB, Lao Women Union (LWU) and Lao Youth Union (LYU), as well as leadership arrangements 14. Improving health facility WASH has become part of cross-sectoral activities for MNCH-N, however, collaboration at the community level has not been concretely implemented. As the sectoral divide at the community level was not present. The WASH project of Bokeo lays out its goal, key objectives and implementation strategy in the Project Design Summary The overall goal states as Improved health conditions resulting from changed and sustained WASH behaviours among children, women, 13 Maternal, Neonatal and Child Health and Nutrition: July 2014 to June A case study of Plan s ECCD and WASH programme in Bokeo Province, funded by Austrlian AID, May

25 and ethnic and other vulnerable groups in 170 selected villages in Pha Oudom, Mueng, and Paktha districts of Bokeo and in Houn district in Oudomxay province by 2016, thereby contributing to Lao PDR s MDGs and development goals for There are four objectives: 1. Improved and maintained good personal and household hygiene, hand washing with soap at critical times, and consuming safe drinking water among community members, particularly among women and children in selected target communities and schools. This is based on the fact that hygiene practices and sanitation standards are low in target villages with high reported incidences of diarrhoea. The rationale is that improving and sustaining good hygiene behaviours will improve health conditions. Therefore, both schools and villages should meet basic WASH standards for measurable health impacts. 2. Achieved and maintained ODF status in 80% of the selected target villages and schools. This objective aims to address low toilet coverage and ineffective usage of toilets is common in many villages. Less than half of the primary schools have adequate school WASH facilities. Therefore achieving and sustaining community ODF status (100 % toilet access) is a critical milestone for realizing the health benefits from having toilet coverage in villages. 3. Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages. Water supply maintenance is often poor and systems do not meet village service delivery requirements. Furthermore, lack of water availability can be an impediment to improved hygiene and sanitation practices. Therefore, improved access to safe water supplies will improve the health and quality of life of targeted communities. 4. Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level. The rationale is that there have been few women in WASH leadership positions and lack of government policies recognizing ODF as a nationallyrecognized sanitation outcome. To achieve these objectives the Program focuses on five key areas of intervention including: Community-led total sanitation (CLTS) in villages and schools, Hygiene Promotion in Schools and associated communities, School WASH facilities and associated community water supply improvements, and Behaviour Change and Communication. The above objectives are to be met through these key interventions. These field level interventions were complemented by a series of research and advocacy activities that Plan and its partners have conducted collaboratively. These include the following: Co-hosting with WSP, SNV, UNICEF, and National NamSaat launching the Lao PDR National Rural Water Supply, Sanitation, and Hygiene National Plan of Action (RWSSH NPA) in September Co-hosting with WSP, SNV, UNICEF, and National NamSaat arranging a workshop in Vientiane on developing operational guidelines for the sanitation and hygiene promotion components of the RWSSH NPA in October Supporting National Supply Chain Survey and Consumer Research initiated by WSP, UNICEF, SNV, Plan and National NamSaat in June & July

26 Co-hosting with WSP, SNV, UNICEF, and national NamSaat arranging the first National Scaling up Sanitation workshop (April 2014) for national, provincial and district government counterparts working in WASH sector and NGO representatives. Contributing by Plan WASH to development of Lao PDR WASH in Schools publication printed in This has been the area of focus that Plan has adopted with its strong trust on the CLTS. Plan is reliant on the community participation and initiatives trying to promote ownership of the development activities by communities and local governments. As stated in the WASH Program document, government agencies and donors began to realise the ineffectiveness of the direct subsidies approach to household sanitation in Lao PDR. Newer approaches for scaling up sanitation coverage in rural areas include Community Led Total Sanitation (CLTS) and active engagement of the private sector. CLTS is currently being implemented in Lao PDR by Water and Sanitation Project of the World Bank (WSP-WB), Netherland International development (SNV), Plan, and World Vision among others 15. Plan has adopted the CLTS approach as opposed to direct subsidies to increase sanitation coverage and increase hygienic practices. The main focus of national level work is advocacy for the provisions under the RWSS Strategy. To this end, the Program increased advocacy work with the MoH and with key partners such as UNICEF, WSP-WB, and SNV and others. The review of national CLTS guidelines and trainers guides as well as a national level Training for CLTS Trainers was conducted in cooperation with the National Centre of Environmental Health and Water Supply (Namsaat), SNV, WSP-WB and Plan Laos. Development of village level water safety planning guidelines and manuals for use on a national scale has been developed in a similar way with the addition of the World Health Organisation (WHO) and CAWST. Implementation mechanism The focus of the intervention was placed at the local levels from provincial to community with most responsibilities rested on the district level. At the district level, Plan established a technical working team called District WASH Teams (DWT). A DWT was established in each district with approval from each District Governor which means these officials were assigned to work with Plan. The DWTs are multi-sectoral teams comprised of technical staff from DESB and DHO as well as members from LWU and LYU, who were responsible for providing guidance and monitoring for construction of school WASH infrastructure, community water supply systems and facilitate CLTS steps and hygiene promotion activities. Although previously implemented as part of a wider Basic Education Program under a MoU with the Ministry of Education and Sports (MoES), learning from this pilot phase, the WASH Program came under a new MoU with the Provincial Department of Health (MoH) in The District WASH Teams (DWTs) are comprised of up to eight members, with an equal number from the District Education and Sport bureau (DESB) and the District Health Office (DHO), with additional members coming from the Lao Women's Union (LWU) and Lao Youth 15 WASH , Project Design Summary, Pha Oudom, Paktha, and Mueng Districts in Bokeo Province and Houn District in Oudomxay Province, Lao PDR. 6

27 Union (LYU). Up until recently (with the signing of the MoU which broadly defines the role of each sector), the roles and responsibilities of each sector/team member had not been documented, and the teams had been first led by a member of the DESB supported by a Plan staff member and this changed under the new MoU arrangements to a member of Nam Saat section of the DHO. The Mobile Clinics team draws mainly from the health offices at the district level though sometimes LWU staff joint as back up support. At the district level, MNCH-N relies on Mother and Child Health Section of the DHO to take lead in the planning, implementation and reporting. For example, Integrated Primary Health Care (PHC) Mobile Clinics intervention (i.e. Integrated Outreach) started with the district undertaking a process of micro-planning where the preparation for the outreach takes place. The Integrated Mobile Clinics activities involved provision of ANC, Vaccinations, Nutrition education and growth monitoring. The implementation involved at least one or two staff of HCs and VHWs in addition to District level government staff and Plan s technical staff A District level steering committee, chaired by the District Governor's Office, provides over sight, meeting six monthly to review the implementation of all Plan supported Programs in the District, including WASH and MNCH-N. The multi-sectoral review of WASH and ECCD found that while Program documents specify a provincial level Implementation Management Committee (IMC), it appears that this has not functioned as intended, perhaps due to sectoral differences being more firmly entrenched at higher levels as well as the fact provincial level departments are relatively distant from implementation in the field. Community/school engagement The village cluster heads (Kumbans), VEDC, village chiefs, and village health volunteers were actively engaged in the hygiene promotion activities at schools and CLTS triggering processes. They also supported school and community organizations for infrastructure development such as construction of toilet and hand washing facilities at schools, community household toilets, and improvements to the existing school/community water supply as may be necessary. Plan staffing The Plan WASH staff, with support from their colleagues working in other Plan Programs in the same communities, ensures that all activities were implemented in the highest quality possible and that all of the support requirements for the WASH and MNCH-N initiatives were drawn from the government counterparts. Both Programmes, at their initial stages had International Programme Managers. Currently, an International Public Health Advisor provides technical support and overall management support to both WASH and MNCH-N programmes while two National managers each supervising the respective Provincial level managers were posted to manage the projects and the coordinators for each program at the district level. 7

28 Table 1: Timeline of key events for both programs MNCH-N WASH Combined Description Pilot Project Evaluation of the Pilot Project Commencement of the current project MOU process with PHD Baseline Survey Program review Project Expansion to Oudomxay, Houn district Project expansion to Saravane, Saravane and Ta Oi districts Jan March An internal review with PHD Evaluation* Jun Jul/Aug Project continuation in 2 districts at different levels of engagement in Bokeo and Oudomxay MOU with PHD expires Jun December Jun 9

29 The overlap of goals and potential of synergy between WASH and MNCH-N had always been recognized by Plan since the beginning of both Programs. However, apart from the baseline survey and the present evaluation, the two Programs evolved independently from each other. Acknowledgement of the fact that integration of WASH and MNCH-N strategies would yield better outcomes was growing with the global evidence and experience that Plan gained through implementing these two sectoral projects. Plan intends to continue to engage with MNCH-N with a focus on nutrition, in a way that is integrated with WASH. As indicated above, an evaluation of the MNCH Program is to be conducted jointly with the MTR for the WASH Program. This Evaluation is expected to provide empirical evidence and recommendations for Plan to be incorporated into the process of integration of the two Programs with a renewed focus on nutrition The evaluation Purpose of the evaluation As indicated in the ToR, Plan International Lao PDR (hereafter referred to as Plan) wishes to have an evaluation of the Maternal, Newborn and Child Health and Nutrition (MNCH-N) Program conducted jointly with a review of the WASH Program. This evaluation is funded by Plan International Australia (PIA). The specific objectives of the evaluation are to: 1. Assess impact and outcomes achieved through the respective projects against the approved project designs, and identify areas of synergy between the two Program. 2. Assess the likelihood of sustainability of the impact and outcomes at the grassroots, district and provincial levels, with particular attention to the capacity of the Provincial and District health care delivery system and linkage to the private sector in sustained access to WASH services. 3. Draw lessons learnt on Plan s role, approaches and its financial investments vis-à-vis the public and private sector actors on Health and Water, Sanitation and Hygiene ( WASH) in Bokeo Province. 4. Assess the alignment of the MNCH-N and WASH Program designs with the Food and Nutrition Security Action Plan (FNSAP 16 ), and contribution of the Programs to addressing stunting. Highlight opportunities and gaps in the current Program designs that are of relevance to future Program development on a WASH-MNCH-N integrated design with a stronger focus on nutrition in Oudomxay and Saravane provinces. This is especially relevant as Oudomxay and Saravane are prioritized provinces in the geographical focus of FNSAP. 5. Provide evidence on effectiveness of interventions and models used in the WASH and MNCH-N Program, and identify elements of replicability and scalability, while recognizing factors to be considered in adaptation to specific contexts. 16 During the evaluation, the government with FAO support commenced a review of the National Nutrition Policy and Plan of Action, drawing from the Food and Nutrition Security Action Plan referred to here. At the time of this evaluation report finalization, the policy review process was not yet complete. 10

30 6. Gather voices and feedback on the two Program from diverse population groups in the target communities. Facilitate participatory, yet critical reflection and learning among Plan and the government counterparts on how changes happen at the community level. The evaluation primarily employed quantitative and qualitative methods. The evaluation followed the assessment criteria set in the ToR including: relevance, effectiveness, impact and synergy, sustainability, scalability, gender, accountability and inclusion (in relation to Plan s child centered Community Development standards). Specific interview guides have been developed by the team with support from Plan Regional Specialists. These will be used with partners, implementers and participants in order to obtain responses to the questions in the evaluation matrix. Desk review During the inception phase we also conducted a review of key Program documents and information received from Plan. Secondary sources of information were also reviewed including key national documents and sectoral documents. Primary Data Collection The Evaluation Team has identified a mix of quantitative and qualitative tools to collect the data needed for the evaluation. Given the large number of target villages, type of activities and complexity of Program designs and monitoring framework, the adopted mixed method would allow the evaluation to capture depths and breaths of the data. This also was adopted in order to resonate with the methodology adopted in the baseline survey. Since Plan has just finished an internal review on MNCH-N raw data from this review was utilized and reanalyzed, where they were relevant. These comprise the following: Quantitative Data A HH survey using a highly-structured quantitative questionnaire was conducted in July Using a multi-level cluster sampling methodology, the survey sampled 378 households living in 20 target villages across three target districts of Bokeo. The sample villages were representatives of diverse ethnic groups, a mixture of good and poor performers, easy and difficult access, full and partial coverage by MNCH-N and WASH programs. The inclusion criteria are as follows: At the villa level: a representative of at least one major ethnic group either with difficult or easy accessibility to a district center or health centre either good or bad performers according to the reported indicators either covered by both or one program (to compared impacts and synergy) At the household level: Households with children under the age of five years old were prioritized. The evaluators learned from Plan s staff that if households with children were not prioritized, many of the questions concerning MHCH-N activities would be left blanked. In these households women were asked to take part in the survey. In the case where households with children could not 11

31 be found, men were asked to take part in the survey. This is to strike for a gender balance of the respondents. The selection of sampled villages was conducted in a participatory workshop with Plan National and provincial managers of WASH and MNCH-N in Plan CO in Vientiane. It was decided to take 20 % of target villages with overlapping coverage (both MNCH and WASH) as key determinant. However, to compare the results, two villages with presence of only MNCH- N were included. The length of the Program presence was also considered when selecting sample villages in order to assess the likelihood of sustainability and functioning of the local government institutions. (For example, target villages where Plan was present for the last 3 years). Within a village, 20 % of total HHs were randomly selected to participate in the HH survey. Table 2 below shows the sample frame for the HH survey and FGDs. Table 2: Sampling frame for the household survey and FGDs Name Ethnicity No. HH No. HH survey sampled No. HH FGDs sampled Meung (MM) 1 Phonsavang Museu Monelaem Akar men and 6-8 women 6-8boys and 6-8 girls Village authorities 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities 3 saychaleun Khmu Longphabath Museu Namkhaleu Leu Donemoun Leu Paktha (PKT) 1 HouayNorkhom Hmong Houaythong Khmu Houaymong Khmu men and 6-8 women 6-8boys and 6-8 girls Village authorities 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities 4 HouayKhot Hmong Houaysaed Khmu Donemixay Khmu Pha Oudom (POD) 1 Phonkeo Khmu men and 6-8 women 6-8boys and 6-8 girls 12

32 Name Ethnicity No. HH No. HH survey sampled 2 Thamphakae Hmong Tinpha Hmong Mokkhatheung Khmu No. HH FGDs sampled Village authorities 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities 5 Namyaokao Leu Houaykoun Khmu Parkhard Lao Phienghad Khmu Total 6 Ethnicities The questionnaire collected demographic information, opinions, and factual data. The development of the tool has been based on previous experience in similar surveys by the Team, and the support from Plan Regional Office. It is important to note that the sampling approach adopted in the joint WASH and MNCH-N baseline survey 2013 was not followed for the following reasons. The baseline methodology was based on the Lot Quality Assurance Sampling (LQAS) methodology which used the supervision areas (SAs) or catchment of a HC/district hospital as a key determinant of a subgroup of survey population. The target populations for the baseline were women with a child <1 month, children 6 23 months, and children months. A questionnaire was developed for each specific population. Adopting such an approach would not allow the evaluation to ensure conditions such as remoteness, reported performances, ethnic group specialty as described above. Using a proportion to size, villages were selected based on a district as a key determinant of a subgroup of evaluation population. By using either SA or district catchment the sample size of the baseline survey and the evaluation was not much different -392 and 378 respectively. Qualitative Data Key informant interviews were conducted at central, provincial, district and communities. Representative of key stakeholders were selected for interview. They have been identified through the project stakeholder mapping Focus group discussions were conducted aimed at collecting opinions from a variety of stakeholders at local level, comprising men and women, boys and girls (separate group) with an attention to ethnic origin and disability inclusiveness and elimination of all form of discrimination. On village levels this included interviews with 24 focus groups from 6 sample villages of three target districts. In each village, the team was introduced to participants to provide them with general information in an introductory meeting chaired by Head of Village in presence of 20 to 36 villagers before they were asked to form groups for FGDs. 13

33 Technical observation: evaluators conducted site visits to representative water and sanitation facilities to examine the work quality, and functionality of operation and maintenance. Observation was made with three HCs using prepared checklists. Data management and analysis Stata software, version 10.1 (Stata Corp., College Station, TX, USA) was used for quantitative data management and analysis. There were 378 respondents participating in the survey. However, after screening 13 cases were eliminated because of conflicts of information in their responds. Of 365 women interviewed, 80.6% belonged to the age group of years old; 48.8 % were from Khmu ethnic group. The average age was 26.8 years (ranged years); a high proportion of women did not attend school (42.7 %); followed by attending a certain grade of primary school (31.2 %) % of women interviewed got married and the age when they married were mainly from years old and years old (41.4 % and 32.3 % respectively). The nutrition status of children aged less than 2 years was evaluated. 279 children were measured 47.3 % of which was female. The average age in month (m) was 12.2 m (SD = 7.0 m, min= 0 m and max 23 m). Data analysis was structured into two components: performance evaluation and qualitative assessment. The performance evaluation was based on measurement of the achievement of the main results compared with the project s initial purposes in term of outputs, outcome and goal. The qualitative assessment assessed the Program designs, approaches adopted, implementation process and monitoring and reporting system. Project achievements were evaluated based on the degree of the realisation of the objectives in terms of relevance, effectiveness, impact and synergy, sustainability, scalability, gender, accountability and inclusion (in relation to Plan s child centered Community Development standards) as set in the TOR. Constrains and Limitations The evaluation was taking place during the two months school break period. The team could therefore neither meet children at school nor observe the school WASH facilities being used. Instead children were met in the village premise and efforts were made to request teachers or school principals, if they lived in the village, to enable the team to visit the schools and observed the conditions of the school WASH facilities. As mentioned earlier, the Plan began in With staff turnover in Plan and partner organizations, it was difficult to obtain some important Program background information that was not documented. For example, the information on the changing proposition about disability inclusiveness in WASH facilities was difficult to obtain. In some cases, it is also possible some information might be government oriented bias. During the field work, government officials, i.e. the district coordinators, accompanied the evaluation team to some of the meetings at village level. While these officials to a great extent facilitated a successful schedule and did not in any way directly interfere with the team s role as an independent entity, possibility of their mere presence having an impact on the outcome of the interviews cannot be completely ruled out. 14

34 It is of note that the difference in the sampling technique adopted in the current evaluation and the baseline survey. The difference made it difficult to directly compare the results with the baseline. This was more so for MNCH-N Program where more comparison was made. Gender and social inclusion was integrated into a gender analysis based on the gender equality principles in the Gender WASH Monitoring Tools (GWMT). However, we could not arrange FGDs by age groups such as young people, middle age and elders as the tools indicated due to difficulty in field work arrangement. We were able to conduct FGDs with groups of women, men, boys and girls separately. In terms of gender, the analysis focuses on four key indicators including gender and WASH roles at household level, gender and decision making at household level, gender and community WASH participation and gender and decision-making at community level. This analysis assumes that the common social norm in all ethnic groups is that women are the bearer of the home chores including taking care of children, cooking, fetching water and cleaning of general environment of the house. 15

35 2. Results, achievement and progress 2.1 Achievements of the WASH program This section presents the achievement and progress of the interventions towards the set objectives. This report presents only the results of the Midterm Evaluation of the WASH Program in Meung, Parktha and Pha Oudom Districts of Bokeo Province. It has to be pointed out that the WASH project suffered from the late start-up due to delay in the signing of the MoU. Therefore, to some extent this delay in MoU singing could have an impact on rolling out of the project activities. Achievements and progress toward specific objective 1 Improved and maintained good personal and household hygiene, hand washing with soap at critical times, and consuming safe drinking water among community members, particularly among women and children in selected target communities and schools As of June 2015, the project has achieved 3 of the 10 annual targets by 100% making those three highly likely to meet the targets set for These include indicator 1.8 % of ODF villages that maintain Ban Sam Saat, Model Healthy Village, SoQ, ODF standards 1, 2, and 3 years after certification, indicator 1.9 % of target villages that have village sanitation and hygiene regulations and indicator 1.10 % of trained District WASH team members undertaking project activities on regular basis in schools and villages. The progress reports did not monitor achievement under indicators The results indicators 1.4 % increase among women and children who know least three critical times to wash their hands (target 80%) and 1.6 % increase among women and children who know at least two reasons that cause diarrhoea (target 80%) were discussed under the section Effectiveness. The indicators 1.3 Number and % of episodes of diarrhoea reported in sample target villages, 1.5 % of reduced costs for medical expenses related to poor sanitation and 1.7 % reduction of number of sick days from work or school resulting from being sick (with diarrhea) were not assessed in the evaluation as the MER review recommended that these indicators were too difficult to record the progress and should be dropped. o Indicator 1.1 total number of target villages meeting Project ODF and HP conditions (officially declared) that correspond to Model Healthy Village indicators related to WASH (No. 1, 2, 7, noting that Ban Sam Saat indicators are included in MHV indicator no. 2-78% (91 villages) of the target (117 villages) has been achieved by June Indicator 1.1, therefore, is well on track and likely to achieve the end of the project s target of 91% of the target villages (128 villages) I N D I C A T O R 1. 1 A N D 1. 2 P R O G R ESS Villages Schools 23 Linear (Villages) Linear (Schools) J U N - 15 J U N E ( T A R G E T ) 91 16

36 o Indicator 1.2 total number of target schools meeting Project and HP (star topics) conditions (Officially declared) that correspond to Lao PDR School of Quality indicators related to WASH criteria and inclusion - 45% of the target achieved DWT members and village authorities interviewed reported that there were three main aspects to difficulty achieving the targets. According to the district staffs interviewed, one reason is that there was generally a low level of awareness of good hygiene practices among a large section of community members therefore they did not build toilets even after attending CLTS process. The view was well supported by the VHCs and VWCs. The second aspect is that many community members were so poor that make them worried about having something to eat than thinking about where to defecate. Some KII respondents reported that the implementation of CLTS activities such as triggering and post-triggering following ups at the community level contributed to the low performance for indicators 1.1 and particularly for 1.2. They were of the opinion that many DWT members lack the facilitation skills that are participatory, the skills that would encourage villagers to build toilets and improved other good hygiene practices. They admitted that the CLTS process relies on the participatory and active engagement methods which many facilitators have limited skills on. It is true that there had been refresher trainings provided to DWT members annually, but many reported that some members were still not comfortable applying the participatory methods. The KII respondents reported that the training they received and the field work where they could put new knowledge and skills in to practice were not frequent enough. It has to be cautious for the analysis of performance for indicators 1.2 though that the testimonies from school officials were not obtained and incorporated into the analysis. In conclusion, the low performance on some parts of the indicators resulted from ineffectiveness of the CLTS implementation by DWTs as well as the low level of general awareness about health issues from the communities. Indicator 1.1 and indicator 1.2 involved behavioral changes to meet the target. The KII respondents reported that the poor success rate is attributed to general poverty among villagers of the target villages that make them unable to afford toilet construction cost. They also claimed that behavioral change takes time. One respondent said many of the villagers only learned to use salt a few years ago, now we want them to use toilet, it is not going to be an easy objective to achieve. The participants referred to some of the target communities that had just resettled from their traditional temporary settlement that moved from place to place. Before, they had little exposure to the outside world and even salt was difficult for them to have access to. They also claimed the issues concerned behavioral change such as values, attitudes, self-efficacy and habits were deeply rooted within the ethnic minority groups who until recently live their life close to the nature. The FGD results also shows that villagers spend a lot of time in the field, they were shy to interact with officials including the evaluators and their cultural beliefs such as their use of traditional healers do not support behavioral change. 17

37 Table 3: The targets and the reported achievements of the objective 1 Indicators Year 1 (Jul 12-Jun13) Year 2 (Jul 13-Jun14) Year 3 (Jul 14-Jun15) Jun 16 targets/# Achieved (% of target achieved) Objective 1: Personal and household hygiene in selected target communities and schools, improved and maintained, particularly by women and children (All indicators cumulative targets) 1.1) total number of target villages meeting Project ODF and HP conditions (officially declared) 1.2) Total number of target schools meeting Project and HP (star topics) conditions (Officially declared) 52* /91 (78%) /53 (45%) 1.3) - 1.7) some of these indicators were assessed in this evaluation and presented in the effectiveness section. 1.8) % of ODF villages that maintain Ban Saat, Model Healthy Village, SoQ, ODF standards 1, 2, and 3 years after certification 1.9) % of target villages that have village sanitation and hygiene regulations 96% 96% 95% 100%/95% 100% 100% 100% 100%/100% 1.10) % of trained District WASH team members undertaking project activities on regular basis in schools and villages 100% 100% 20 (10 f), Active 18 (9 f) 100%/100% *included target villages. Under indicator 1.1, one of the key important conditions is the consumption of HH treated/boiled water. To this end, women are still facing disadvantages. For FGD respondents, safe drinking water almost equaled boiled water. They also admitted that boiled water is only available at home (villagers reported that they did not bring with them boiled water to the fields). When considering the fact that women spend more time than men in the field: collecting firewood, weeding in the upland rice field (hai), or searching for food in the mountain/forests, they are likely to miss out on boiled water. As shown by indicator 1.2, the number of schools meeting the star topics conditions had been under-achieved. This meant that children are not able to have access to improved sanitation and hygiene while they are at schools. Given the challenges, some of which are beyond this project s sphere of influence, the targets set by the project for indicator 1.2 can be too ambitious. It was reported by some children in the FGDs that in some schools teachers locked up the toilets keeping it for their own use, claiming children did not keep the toilets cleaned. This was the case in two out of six target villages visited by the evaluators. This has negative impact on children s access to sanitation facilities. VEDCs reported that even though teachers were oriented about toilet uses, some still keep the toilet use to themselves. The key reason given was children make the toilet dirty. 18

38 It was reported that schools across the 3 project districts have functioning gender-segregated toilets that are being used by students. However, due to the school vacations, we could not observe how children use the WASH facilities in schools (evaluators managed to visit three schools). From three schools what was observed was that the toilets have separate rooms for boys and girls. They all looked clean with water available. The latrines were not modified for disability inclusiveness but looked child friendly: the water storage and taps located at the level easy to reach by a child. There are hand-washing stations built with three levels of heights suitable for different age groups to use. The school premise was kept clean particularly around the building and toilets. Even though there were places made particularly for putting soap, there was no soap there, because it was the vacation period. It was evident though that soap had been in use in the stations which was confirmed by the accompanying teachers and village authorities. Since hand washing with soap (HWWS) is a new concept in the area, the evidence of HWWS in this hand washing station shows the presence of enabling conditions for behaviour change that was taking place as a result of the project intervention. Achievements and progress toward specific objective 2 Achieved and maintained ODF status in 80 % of the selected target villages Interviews with the project staff revealed lack of clarity about indicator 2.1 and 2.2, particularly around their difference from 1.1 and 1.2. However, document review clarified 2.1 as the number of places where both school and village was ODF. As can be seen from Table 3 below, by June 2015, the project has achieved ODF status in both school and village in a total number of 46 villages in Bokeo, 95 % of the June 2016 target according to the monitoring reports, but this could be inaccurate due to the fact that there was no clarity among project staff about the indicator. Similarly, under 2.2, what has been reported is the number of schools achieving ODF (a total number of 53, which is the same as 1.2) rather than the number of WASH facilities meeting the Ministry of Education standard which is found to be the meaning of the indicator 2.2. Indicator 2.4 Number of additional people with increased access to basic sanitation (no. of HH toilets x 6 persons/hhs avg.) has increased steadily moving toward the 2016 target. In year 1, the project has enabled 5,694 people to have access to improved sanitation, this increased to 9,912 by June 2014 and 15,804 by June The project has achieved 88 % of its 2016 target of 18,000 people. By this rate of performance the project is likely to over-achieve its target by the end of the project. For the rest of the achievement under this objective see Table 3 below. In summary, the project had made significant progress toward objective 2, only about % remains to be done to achieve the 2016 targets. However, the target regarding ODF village remain off-track in terms of meeting the targets. But there are many factors contributing to this under-performed indicator. One of the reasons is the over-ambitious target as mentioned earlier. The project progress reports show that by June 2015, the ODF retention rate has been 95% of total ODF villages. During the FY15, the slippage rate is 4 to 5 %. By June 2013, out of 32 ODF villages, 7 villages had slipped, losing the ODF status. By June 2015, 3 of these villages remained non-odf. This means that some villages that lost ODF status had been reinstated the ODF status as a result of those slipped HHs rebuilding their toilets. The KII respondents 19

39 reported that the slippage in their villages mainly occurs with the dry-pit type of toilets structures of which are simple and prone to extreme weather events such as heavy rain. Based on the narrative reports of the project slippage have also occurred due to termite damage of local woods being used for toilets. Table 4: The targets and the reported achievements for the CLTS intervention component (source: annual narrative reports ) Indicators Year 1 (Jul 12-Jun13) Year 2 (Jul 13-Jun14) Year 3 (Jul 14-Jun15) Jun 16 targets/ % Achieved Objective 2: Achieved and maintained ODF status in 80 % of the selected target villages. (All indicators cumulative targets) 2.1) Number of officially declared ODF communities (both school & households ODF) 2.1a) Number of village toilets constructed 2.2) Number of (Plan) school WASH facilities in target villages that meet MoE standards (toilets and hand washing stations) repaired or constructed (AND FULLY OPERATIONAL) /95% 979 1,648 2, /100% 2.3) Number of buildings and facilities made accessible to people with disabilities (mainly applied to School WASH facilities) DESIGN modification 3 redesigned 25/12% 2.4) Number of additional people with increased access to basic sanitation (no. of HH toilets x 6 persons/hhs avg.) 5,694 9,912 15,804 18,000/88% 2.5) % toilet coverage in all target villages to date (including those triggered in ) 2.5) % toilet coverage in all target villages to date (including those triggered in ) (CUMMULATIVE TARGETS) 76% 81% 81% 90%/84% 67% 69% 69% 90%/72% 2.5) % toilet coverage in all target villages to date (including those triggered in ) (CUMMULATIVE TARGETS) 47% 62% 76% 90%/84% 2.5) % toilet coverage in all target villages to date (including those triggered in ) (CUMMULATIVE TARGETS) 72% 90%/ 20

40 2.6) % of poorest (in triggered villages) having access to toilet facilities 17%- 42% 53-59% 90%/80% 2.7) % Increase of women making decisions in household about toilet provisions 72% joint baseline 2% 90%/ During interviews with the FGDs and KIIs respondents, the team found that most villagers prefer the pour-flush type of toilets because the limited space at their home premise means that they have to build toilets close to their house. In cases where proper dry pit toilets were not built and maintained well/hygienically, people reported that it cannot contain the unpleasant smells. So the pour flush were preferred over the dry pit. Facilitators reported that it is quite a dilemma for them to promote low cost dry pit toilets among those low/no income communities while at the same time have to promote a well/hygienically built toilets. This also indicates that facilitators often tried to influence the toilet type. More discussion on effectiveness of CLTS and progress toward targets will be presented in the EFFECTIVENESS section. Indicator 2.7 shows huge difference between baseline and evaluation findings. In the baseline, 72% of the women reported they made decisions concerning sanitation provision in the household. In this evaluation only about 2% of the respondents reported wife made the decision concerning toilet construction in the household. This difference may come from the fact that the evaluation asked the respondents to choose between husband, wife or together. This may lead to the difference in the results. Achievements and progress toward specific objective 3 Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages Under this objective, key activities include construction of Community Water Systems CWSs, training on water safety plan, implementation of the village regulations and establishment and training of the village water committees. There were 879 additional persons (537 female) who have been provided with access to safe water. This result has been achieved by the construction of 2 additional CWSs in Year 3 supported by ANO in Houaysoua and Tinpha villages in Bokeo. By June 2015, a total number of beneficiaries that Plan WASH Program provided access to safe water reached 2,754 (including 1,460 female). The achievement of 48 % of the 2016 targets as shown in Table 4 below reported only ANO funded CWS (2 CWS FY15). However, the achievement of the 2016 target will be met by CWS supported by other projects. For example, there are 2 other projects on going with 10 village water system. Training on water safety planning was conducted for a total of 26 participants (including 2 female) supported by UKNO. The participants came from DWTs, water supply officials from all 5 districts of Bokeo province including the two non-target districts and the environmental health and water supply centre (Central Nam Saat). The KII respondents reported that the 21

41 knowledge and skilled gained is being used by the participants in the villages where the new water systems are being built. See Table 4 below for more details. By June 2015, 8 villages implemented village regulations that include catchment protection in line with Operations & Management (O&M) and Water Safety Plan (WSP) guidelines. The village authorities interviewed reported that the training and implementation of the village regulations and the water safety plan contributed to the development of health conditions that contributed to meeting the conditions of MHV, the initiative under MNCH-N Program. All eight water committees in Bokeo were established and regularly collected fees for maintenance of CWS according to an agreement in each village. The arrangement in terms of maintenance fund for CWS varies from village to village. The most common practice found from interviews with village authorities is that they designated water use groups organised by public water points. The HH in the catchment of a particular water point was held responsible for the maintenance of that water point. Table 5: The targets and the reported achievements for the WASH infrastructure construction Indicators Year 1 Year 2 Year 3 Jun16 targets/ (Jul 12-Jun13) (Jul 13-Jun14) (Jul 14-Jun15) % achieved Objective 3: Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages. (All indicators cumulative targets) 3.1) Number of people provided with increased access to safe water 250 / 118 f 1,875 / 923 f 2,754/1,460 f 10,000/48% 3.2) District and provincial authorities water safety plan monitoring system in place and updated regularly Not started Not started 26 from other 2 non-target districts 21/26 (124%) 3.3) % Village regulations include water catchment protection in line with O&M and WSP guidelines 3.4) % Water committees collect dues from user households regularly (depending on village decision) 3.5) Number of safe, sufficient and protected water supply points constructed or rehabilitated /24% /62% 1 CWS 2 CW 8 CWS 17/47% 3.6) Number of village committees that have training on water safety planning (including villages villages) 5 village water committees 7 village water committees 13 village committees 21/21% 3.7) % of poorest participating in water supply decisions in their villages Not reported this quarter. Not reported this quarter. Not reported This quarter 22

42 Achievements and progress toward specific objective 4 Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level This objective focuses on key good governance principles such as participation in the decision making concerning WASH in schools and communities, satisfaction of end users with the management and governance of WASH system or activities established or initiated by the WASH Program and women and marginalized population contribution to such management and governance. The first indicator of this objective is concerned with participation in decision making in the WASH in schools and communities. However, monitoring on this indicator in school is difficult and has not been done recently. The evaluation surveyed participation in the WASH governance and management at community level with 87% of the respondents reported they were actively involved in the decision making concerning WASH facilities in the community. By June 2015, Behavioral change communication (BCC) activities had not been fully implemented by the district WASH teams. On average, 50 % of planned activities had been implemented. For example, 33 % of BCC activities were implemented during Year 3 or FY 2015 under ANO grants support. It is important to note that the percenrage had dropped from 75 % in year 2 to 33 % in Year 3. This is a significant decline within one year timeframe. It was reported in the progress reports that the reason for under-achievement of the target is time constraints faced by the PHD and DHO staff. The time constraints faced by staff should not be that much different over one year given there was no reduction in the number of government staff during this period. According to the KII results (presented in the Effectiveness section), the reasons for many activities being delayed are many including high staff turnover rate for both Plan and the implementing partners, ineffective communication and coordination between project staff and the government staff, delay in the process of activities planning, implementation and reporting. In terms of women participating in village meetings related to WASH, according to GWMT findings, it was shown that even though all women had been invited, many women in rural areas were not able to participate fully in the meetings because of language barrier (most meetings had been conducted in Lao Lum and there was no particular method to facilitate the meetings with participants from a diverse language background) and they are shy to share their opinions. For women taking on leadership (decision making) roles, 50% of District WASH team members are women in Bokeo. 14 % of those assuming leadership (decision making) roles at the local level (village committees) are currently women although even they may not be confident and have equal status to the male counterparts. The first training about disability, inclusion, and equity for Plan staff at Bokeo PU was conducted under a partnership with the Lao Disabled People Association (LDPA). The training was held for 4 days by 3 (2 female) trainers from LDPA (one trainer is a person with disability). The total number of participants was 21 (9 female), this included 5 staffs from WASH team (1 female). KII respondents reported that they understood more about comprehensive categories of disabilities and were able to recognise different types of disabilities exist in the project areas. They claimed to have had encouraged participation of PWDs in the project activities. However, they admitted they did not have specific technics/skills for working with PWDs. See more details achievement in Table 5 below. 23

43 From the KII interviews, it is clear that the project implementation partners including provincial and district officials were actively engaged with the project activities. Many activities of the projects including those under MNCH-N program were in their plan but lack the budget to implement them. The government at all levels said that they were cooperative with development partners because they had access to financial and technical assistance by doing so. Moreover, the government staff at district and HC levels reported that they gained hands-on experiences from many technical/practical trainings. The government staff admitted that while they were able to put what they have learned from the project into practice by the project funding but this could finish once the financial support ceased with the project s exit. Table 6: The targets and the reported achievements for the WASH governance Indicators Year 1 Year 2 Year 3 Your 4 (Jul 12-Jun13) (Jul 13-Jun14) (Jul 14-Jun15) (Jul 15-Jun16) Objective 4: Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level. (All indicators cumulative targets) 4.1) End-users expressing their opinions and are active in decision making in schools and communities 4.2) % of activities in District BCC plans achieved by DWTs 4.3) Level of satisfaction among end users with WASH governance in their villages, toilet and water supply provisions 4.4) Number of women (% of total participants) expressing they have been invited and participate in village meetings related to WASH 4.5a) % of women on District WASH teams 4.5b) % of women assuming leadership (decision making) roles at local (village committees) village CLTS committees 4.5c) % of women assuming leadership (decision making) roles at local (village committees) Village water committees 4.6) Total number of women, especially caretakers, provided with gender awareness raising/training that enables them to better participate in community processes 88% 87% 80%/110% 50% 75% 33% 100%/33% n/a n/a n/a 80%/ 87% 33% 90%/37% 60% 60%. 50% 60%/100% 22% 50% /40% 14% 31% 50%/62% 11 women No training /0% (target for both UDX and Bokeo) 24

44 4.7) Number of men provided with awareness raising/training on gender issues and women s equal rights 4.8) Number of professionals trained in disability inclusion n/a /0% 2000/0% (Target for Bokeo?) n/a n/a 21 (9 female) 25/0% 2.2 Achievements of the MNCH-N program This section evaluates and discusses the achievements of the specific objectives and outcome of MNCH-N Program in Bokeo. Achievements and outcome for specific objective 1 (SO1): Contribution to the development of a sustainable primary health care system in Bokeo province Strengthening the Health management Information System The MNCH-N Program successfully contributed to improving the Health Management Information System (HMIS) through strengthening the capacity of PHD and DHO staffs. These included provision trainings in computer skills. In this regard, two training courses were provided. The first was organised in Pha Oudom in 2012; 25 DHO staff (5 women) attended. The second was held in Paktha in 2014; 17 DHO staff (8 women) attended. Plan also supported the DHOs and PHD in compilation of integrating village and health centre level family/patient files including disease specific statistics, in three target districts to facilitate vital information gathering and storage. Capacity of the PHD and DHO staff regarding this work had been significantly improved. The new obtained knowledge and skills were usefully applied to the staff routine job. The site observation conducted during the field visit found that DHO staff were not comfortable using the computer; however, they were able to use it for their daily work and stored key health information properly as a result of support by Plan. At health centres, the data files were wellarranged and stored and important information for target communities were available. For example, information about a number of target children for vaccination, monitoring growth, and children U5Ys were recorded and updated regularly. DHO and HC staff interviewed commented that the trainings were very useful and applicable to their routine tasks. The increased knowledge and skills facilitated their job immediately such as micro planning process which took place every month. At the village level, village health committees and village heads were provided with trainings on vital statistics and other health related data recording and reporting which fed into the national system. The data concerning the conditions of MHV was prioritized because MHV was known as a part of village data statistics that facilitate village committees as well as the local government institutions in improving the health and hygienic practices in the communities. MHV is the government initiative that Plan has been supporting where Plan also presents, at the moment in Paktha and Meung districts. 25

45 Table 7 below shows that initially 10 villages were targeted and 04 out of 10 (40%) achieved MHV status and 06 villages were under monitoring. In 2014, 24 villages were targeted, 19 out of 24 (79.2%) achieved the status and 05 villages were in the process. The rate of achievement increased significantly from 40% in 2012 to 79% in 2014, an increase of almost two folds in two years. The majority of the KII respondents believed that the integration of WASH and MNCH-N program activities in the same village gave a synergistic effect which accelerated the target villages to achieve the MHV status. This was due to the two programs delivered the same or similar health and hygiene messages. The integration of the programs will be discussed in greater depth in the section on Impacts and Synergy. Table 7: Achievement of establishing of MHV in three target districts FY Target village/s Achieved MHV MHV In process 2012/ (40%) / (79.2%) 05 However, assessment of the MHV s conditions remains a challenge. It was reported that the assessment happened ineffectively. The KII respondents admitted that they (leadership within the provincial government) were preoccupied with the mandate of meeting the quantitative target. This often led to compromise of the conditions during the assessment of the MHV indicators 17. The indicators in the government s MHV evaluation framework that involve some subjective evaluation were the most problematic. The respondents said that assessment of indicators such as indicator 1 Eating, Drinking, and Living Clean was difficult and the assessors often gave a pass assessment assuming their management wanted it that way. Strengthening district, health centre and outreach management To ensure monitoring and supervision in communities, Plan purchased 02 four wheel vehicles, 01 motor boat and 19 motorbikes. This allowed the government counterparts and Plan staff to have all year round access to the communities. A quarterly meeting and annual meeting was help and attended by PHD, DHO and Plan staff. The meetings were meant for implementing staff and the government counterparts to reflect on the project progress, identified issues/problems and collectively addressed those concerns/issues as they arisen. A study visit to Xayaboury province was arranged, supported by an Australia Children s Aid Project. Plan had supported a series of reviews and planning meetings as part of the project implementation. So far, 17 meetings were held (3 annual meetings, 10 quarterly meetings and 4 orientation meetings). Total participants attended were 712 (256 women). Table 8 below shows a high number of staffs attending the meetings. But these meetings often took 17 MHV Indicators: 1) There is Eating, Drinking, and Living Clean; 2) There is a clean Latrine; 3) Environment is clean; 4) There is vaccination according to MoH standards; 5) Pregnant women have access to MCH services in the local Health center; 6) Elimination of mosquito breeding areas and all persons sleep with mosquito nets; 7) There is year-long access to safe water; 8) Animals are kept in pens away from the house; 9) There is knowledge on TB prevention; 10) There is knowledge on HIV prevention. 26

46 place behind the schedule. KII respondents reported that health staffs had many responsibilities and it was difficult for them to be available at the same time for the meeting. In other words, there are too many meetings that demanded too many people to attend. In fact a monthly DHO meeting was organized at DHO and chaired by DHO director/deputy director, with a representative of each health centre within its catchment areas attending the meeting. In order to address the issue of too many meetings the PHD and DHO should incorporate a quarterly meeting in every third monthly meeting of DHO. During this meeting, the program works can be discussed. The MNCH-N Program Plan staff regularly reported progress to headquarter. The government counterparts at the district level reported quarterly and annually to PHD. In turn, PHD reported to the MoH on six monthly and annual bases. However, KII respondents at the central level reported that the core concerned institutions at the central level such as the Nutrition Centre and Ministry of Education and Sports claimed did not receive any reports concerning the MNCH-N Programs activities in Bokeo. In principle, the PHD must report monthly, six monthly and annually to its vertical and horizontal lines. Reporting to its vertical lines means reporting from VHVs to HCs. Then the HCs report to DHO to the inter-hc meetings held at DHO that chaired by DHO director, DHO to PHD through inter-dho meeting held at PHD that chaired by PHD director. PHDs report to the MOH (concerned departments and specialist centres: nutrition centre, MCH centre, etc.). Reporting to its horizontal line means reporting to the district/provincial inter-sectoral meetings (Education, Health, Agro-Forestry, Social Welfare, Information and Culture, Transportation and Communication, etc.) chaired by district/provincial governor. Table 8 Number of annual and quarterly meeting and number of staffs attended Item Number of meeting and its participants No meeting No staff F No staff F No staff F Total No staff F Orientation meeting at PHD (1) and DHO (1 each district) Quarterly meeting in 3 districts Quarterly meetings held in 3 districts six monthly meeting held at POD Annual meeting held at POD Annual meeting at PHD Annual review meeting at PHD Quarterly meeting held in 3 districts Total

47 The findings above show that there is some conflict between the principles and what really occurred in practices in terms of reporting and communication. If the PHD reported to its vertical line the core concerned institutions would have been informed about the progress of the program. Yet they were unaware of the progress. This could mean that PHD did not perform its functions properly. If they did report to its vertical line, then the reports might not have been used as should have been the case. Improving the quantity and quality of human resource for health and nutrition The target of improving the quantity of human resource for health and nutrition had been achieved. Table 9 below shows that several training courses on MNCH-N were organized for PHD, DHO, HC and village facilitators/vhv between 2012 and 2014, with a total number of 898 (387 women) attended. In addition, the program had successfully trained 25 women Skill Birth Attendants (SBA) drawing from ethnic and indigenous high school graduates by using another funding source to complement the activities in this project. They were now working as volunteers in the villages and they would be recruited by the government as health staff next year. Plan continued to support 13 students and they were currently undergoing the course in nursing schools and 11 of them would become SBAs. Building a cadre of SBAs, especially from the ethnic minorities, is responding to the need of the Bokeo province. Even though there was some improvement in terms of quantity, development of the quality of health services remained a challenge. The evidence from the household survey indicates that the ability of DHOs and HCs to take ownership of the program s activities is limited. Some health centres lacked basic medical equipment and medicine (sometime has no medicine). They have not enough scales and measurement equipment for growth monitoring. Table 9 Training course organized between 2012 and 2014 and number of participants attended the training course No Training course No staff F No staff F 1 Training for PHD and DHO FGD facilitators for conduction of PCA in community, trained facilitators conducted FGD with target community members and organized meeting with government counterpart of three districts Sub-Total MNCH and Nutrition activities 1 Long term training course for midwife (SBA) in Vientiane Technical Health College for 2 year Long term training course for midwife (SBA) in provinces Training for PHD, DHO and HC staffs: 3 4-days training of trainer (TOT) on Parenting Orientation concerning MNCH-N for PHD and DHO staffs Training on nutrition for PHD and DHO in 5 districts to be trainer for HC and VHV yes 28

48 5 TOT on basic nutrition and nutrition treatment from all 5 DHO, expected to be trainer for HC and VHV 5 TOT for Micro planning training by 3 main trainers from EPI and MCH center Training on micro planning for HC staff from 19 HC Training for DHO on how to prepare micro planning in terms of integrated MCH package services Training on cool chain system management for DHO Training on cool chain system management for HC staff 6 2 Training for village facilitators and VHV: 10 Trainer on Parenting Orientation continued to train in 57 villages of 3 districts days Training on Parenting Orientation Modules for village facilitators days training on Parenting Orientation modules for VHV Training for VHV on Positive Parenting Program yes 14 Training on positive parenting module for VHV Sub-Total Health Management Information System 14 Training on computer and Microsoft Words for DHO staff of Pha Oudom district by trained DHO and Plan staff 25 5 knowledge increased from 49.9% % 15 Training on computer skill for 2 days for DHO and HC staff in Paktha district Training on MHV components for 6 VHC 38 7 Sub-Total Total During 2013 and 2014, Plan organized three study visits to similar health programs in order to exchange experiences and learn from one another. These are: Health unit of Namtheun 2 electronic company in Khammouane province, there were 10 district health staffs, 2 PHD staff and 3 Program staff and 4 from other districts like 29

49 Tonpheung and Houaxai districts joined this visit. The visit focused on integrated health centre, integrated MNCH-N package outreach, and quality of services; Xaiyaboury district, 20 participants from model health centre, district health staff, PHD, MNCH-N coordinators attended in this visit; and 2014 the study visit was held in Louangnamtha province the UNICEF project area. MNCH team did observe the micro planning training and learnt a lot about well preparation of planning. Overall, Plan made a significant contribution to the development of the PHC system in Bokeo province through the improvement of capacity of health and none health staff therefore it improves the PHC system in this province. Achievement and progress for specific objective 2: Promotion of universal access to quality health care for the families of three Bokeo districts The desk review s findings show that the Plan s MNCH-N Program had supported the routine outreach activities plus growth monitoring, mobile clinics integrated with EPI, quarterly health days, health promotion days, Measles-Rubella campaign, children days, nutrition education and food demonstrations, and implementation of parenting orientation in communities. As shown in Table 10 below, a total number of health events had been increasing each year such as 01 event in 2012, 03 and 08 in 2013 and 2014 respectively. A number of villages covered by health events had also been increasing each year such as in 2012, 2013 and 2014 the health events covered 16, 58, and 329 villages, respectively. A total number of beneficiaries reached 69,334 persons. It is important to note that the number of beneficiaries had increased from 3,900 in 2012 to 11,459 in 2013 and 53,975 in In three districts, the target number of children under five years (child U5Y) was 11,129 and women of reproductive age (WRA) were 17,112. It was reported by the Program that the total number of direct beneficiaries such as children U5Y and WRA who benefited from health events held in communities was 15,052 persons and 24,214 persons (noted: two health events did not disaggregate child U5Y and WRA). Number of direct beneficiaries accessed to each type of delivery health services during the health events in villages was summarized in the Table 9 and Table 10 above. A number of children U5Y vaccinated, received deworming and vitamin A had been increasing gradually throughout the period from 2012 to 2014 (see Figure 13). Children U5Y had been monitored concerning their growth. A number of women of reproductive age received tetanus toxoid (TT) vaccine increased from 285 in 2012 to 498 in 2013 but it was not observed in 2014 due to the information was not made available for some health events reports. Pregnant women received ANC services had increased over time such as 60, 273 and 296 in 2012, 2013, and 2014, respectively as well as for PNC service a high number of mothers received the PNC visit (Figure 14, Table 10). These achievements are cornerstone of improvement of maternal mortality rate and infant mortality rate in this province and also in Laos therefore it has contributed to achieve the MDG1, 4 and 5 as planned. 30

50 Table 10: Summary outreach activities and health events held in communities between 2012 and 2014 and number of beneficiaries attended Activities Total Village (152) Total Beneficiaries attended Direct beneficiaries Children U5Y (in 3 districts 11,129) WRA (in 3 districts 17,112) In 2012 Quarterly Health day in ,900 3, Sub-total 16 3,900 3, In 2013 Quarterly Health day in , ,643 Outreach activities in , Sub-total 58 11,459 3,925 7,241 In 2014: Mobile clinics integrated with EPI 47 3,544 2, Mobile clinics integrated with EPI , ,393 Outreach activity plus growth monitoring 139 2,482 1, Six times health promotion day 4,130 na na Two times Health promotion day Nutrition education and food demonstration Measles-Rubella campaign for children 9m to 10 years 18,597 na na Children day 13, Sub-total ,975 8,010 16,190 Total ,334 15,052 24,214 31

51 Figure 2: Number of children accessed to health services during health events Figure 3: Number of woman of reproductive age (WRA) and pregnant woman (PW) accessed to health services during health events The results from the household survey show, that most children accessed vaccination in village such as 95.5% accessed to measles vaccine whereas the baseline data was 69.2% and 97.3% accessed to DPT3-Hib-HepB3 whereas the baseline data was 65.2%. The access to ANC and PNC care had been considerably improved such as ANC visit 4 times was 47.2% (baseline data was 26.6%), PNC visit within 7 days 58.3% (baseline data was 8.7%), PNC visit within 6 weeks was 35.1% (baseline data was 10.5%). Plan also supported the improvement of cold chain system management and immunization training. Plan together with government counterpart organized the concerned training to 15 (7 women) health staffs in 2014 (9 DHO staff (5 women) and 6 HC staff (2 women)). It is very important activity to support the MOH in terms of improvement vaccination coverage among children and women in remote areas where it is hard to access. Overall, the evidence above indicates that the MNCH-N Program has achieved the target in terms of the number of direct beneficiaries to access to health care services in communities: vaccination, deworming, receiving Vitamin A, monitoring child growth, nutrition education, family planning, PW received tetanus toxoid vaccine, ANC, PNC. This achievement was not identified in previous years before implementing MNCH-N Program in Bokeo (report in 2010). The Program significantly contributed to improving the access to basic health need among 32

52 rural remote population particularly WRA, mothers and children not only in Bokeo but also contributed to the overall achievement of Lao PDR. The KII results show a mixed picture of results in terms of quality of services provided. KII respondents both Plan and government staff reported that they were very satisfied with the mobile clinics and outreach activities. The strengths of this initiative included the following. It was built on the current government program thus there was high commitment and encouragement from the government side. Involvement of HC staff and VHVs made the implementation smoother: they were familiar with localities and the people, their culture, rituals and general life conditions of the target communities. Nutrition education and food demonstration was very effective. The district staff believed it could be used us awareness raising strategy because it allowed communities not only to see but to taste the food as well. In addition, it did not need a lot of talks (in Lao language) which was difficult to understand for some ethnic groups particularly women. Almost all of the respondents shared the view that the program could have allocated higher budget for this activity. However, they were not sure how effective the health messages were absorbed by the participants of the events particularly during the health promotion day. The staff claimed that for the events such as the health day, the evaluators would not be able to properly assess unless it was held more frequently than how they were doing now. Table 11: Summary of direct beneficiaries accessed to each type of health services in communities between 2012 and 2014 Type of health services Direct beneficiaries Children under five years Total Child vaccination ,391 6,499 Child received deworming 380 1,288-1,668 Child received vitamin A 197 1,372 1,092 2,661 child monitoring growth - 3,357 3,544 6,901 Sub-total 782 6,920 10,027 17,729 Women of reproductive age - WRA received TT WRA received TT WRA received TT PW received TT PW received TT PW received ANC ANC visit >= 1 time ,182 ANC visit >= 4 time

53 PNC visit Family Planning Health check and basic treatment ,325 Nutrition education and food demonstration Sub-total 783 2,832 2,641 6,256 Total 1,565 9,752 12,668 23,985 Note: Number of beneficiaries accessing to each health services in some health events was not available Achievements and progress toward specific objective 3 Engagement of Bokeo communities for health by seeking their advice and encouraging local reflection, prevention and promotion activities Plan had successfully completed a Participatory Community Assessment Analysis (PCA) in three target districts after the MOU was signed in The PCA aimed to assess the health problems and needs in communities and to identify the potential actions and local resources to address those problems and needs. This approach was used specifically with individual, families and communities to initiate a process of empowerment among women, their partners, families and communities. The government partners, PHD and DHO staffs, also benefited from this exercise as they had been trained and had a chance to learn by doing particularly on how to facilitate the FGD and capacity to conduct the FGD with community members (WRA, mother, mother in law, husband, village leader and VHV). Community members in each cluster had taken the opportunity to share their views, worries and wishes concerning the MNCH-N problems and needs. The staff interviewed believed that PCA was an effective approach because it gave an opportunity for both health care providers and users to meet and discuss the common health problems and needs in communities. In addition, this participatory approach gave the opportunity to remote rural people to express their opinion so that they would not feel neglected by society. They could initiate some preventive health activities based on their lay knowledge. Based on the PCA assessment, the program was designed with four appropriate interventions including: 1) Providing capacity of care at the household level for pregnant women, newborn and children. This intervention was designed to address the low rate of PW attending the ANC and delivery at health facilities. It was meant to address the root cause of the low rate problem such as shyness, traditional beliefs and financial issues. In addition, they were not aware of the important of health services and the consequences of not using the services. It was reported that PW and postnatal women work hard and the family members thought that it is a normal way of life. Food taboos are still widely practiced in the community; 2) Improving awareness of community of the right, needs and problems related to pregnant women, newborn and children. This intervention addressed the issue of premature marriage. Many young girls got married and pregnant before the age of 18 years old. FGDs participants said that families needed more labor so they encouraged their daughters to get married early. 34

54 This has also resulted in school drop outs. Women did not seek care in health facilities. Some sought treatment from a shaman in traditional ways. Women groups reported that husbands did not care for their pregnant wife. For example, they said that when pregnant women wanted to go to the health facility to seek ANC, husbands went to rice fields or forests. This is particularly important for women in rural areas as they could not travel to health facilities alone due the road conditions and long distance of travel; 3) A linkage for social support between health services and communities. this intervention responded to the findings of PCA which indicated that villagers could not access to health services due to the long distance of traveling and communities could not afford for the fee of transportation; and 4) Improving health services. The evidence of PCA indicated that the services were available for mothers only on Wednesday. In addition the health facility lacked of human resources and medical equipment that are challenging for mother health child services. In summary, the program has achieved the specific objective 3 through the application of PCA assessment. Although health issues for villagers remain less important, health care and treatment seeking are becoming better (field KII respondents). In regards to this change the evidence from desk review revealed that the number of children who vaccinated, received deworming and vitamin A during the health event held in communities has increased over time (Table 9 and 10), the HHS indicated that PW had attended ANC 4 times and postnatal women sought PNC within 7 days and within 6 weeks more and more (Figure 13). In addition, interviewed KII respondents stated that some health staffs were enthusiastic to perform their tasks in community such as health centre staffs asked to be leaders the health event by themselves since they felt more confident after attended training. Interviewed PHD staff really supported this initiative from Plan. 35

55 3. Relevance 3.1 Relevance of the WASH Program The WASH project in Bokeo was designed based on the premise and best practice that the availability and use of improved water and access to sanitation in schools and communities will contribute to improvement in health and nutrition conditions among women and children. SLTS was drawn on the WASH in schools model of UNICEF that aims to bring safe drinking water, improved access to clean sanitation facilities and improved health conditions to schools. 18 Relevance of the project design The Evaluation concludes that the project is relevant as indicated below: - The CLTS and SLTS approach have been well tested in Laos where the alternatives such as direct subsidies from external organisations have been proved failure because it has not led to effective use of sanitation facilities and sustainable behaviour change of key hygiene behaviours. - Adoption of this approach has been endorsed by the government as stated in the National Plan of Action for RWSS and this has been confirmed by KII respondents. - The targeting and prioritization of the interventions are appropriate as the project target rural areas where the toilet coverage and access to improved hygiene practices were low. The project was relevant in terms of responding to the need of the community. The sanitation situation in the three target districts for , Pha Oudom, Mueng and Paktha in Bokeo reflects national trends of low coverage in rural and poor districts. Among the three districts, sanitation coverage rates are below national target rates, especially when consider actual usage. Open defecation rates are still high, despite the provision of toilet materials to village households. 19 Table 12: Summary of estimated water and sanitation conditions in the 4 target districts Coverage estimates 2012 Pha Oudom Mueng Paktha School WASH facilities 38% 67% 40% School HP minimal minimal minimal Water supply coverage (reported) 85% 100% 94% Existing water supplies with low or no functionality (reported) 29% 10% 42% Toilet coverage (estimated) 39% 59% 60% Effective use (estimated) 31% 47% 48% Source: WASH program document, Practice Brief, SNV Laos, Water, Sanitation & Hygiene sector, August Plan Laos, Sanitation and hygiene practices observed in 9 villages in Bokeo Province. November

56 The selection criteria of the target villages, however, are quite ambitious with a number of selection criteria can be very difficult to meet. Furthermore, some criteria are subjective and difficult to assess. Criteria such as villages with strong leadership and fully committed to the process can be very difficult to assess as the strong leadership and good commitment are subjective and cannot be assessed easily at the outset. The same hold true for an active and enthusiastic school director with good coordination with community. Most importantly, the villages and schools to be selected should have functioning and adequate water supplies 20. The condition 5 of new target villages to consider stated in the WASH program document. Plan staff interviewed admitted they might not get any target villages if they follow all 10 criteria strictly. To be discussed in the effectiveness section, however, KII respondents reported that they could not manage to follow up and support post-triggering process at the community level because they worked in too many villages. Water supply system rehabilitation and construction The initial design of the project focused on the improved sanitation and hygiene behaviours in schools and host communities including use of toilets and hand-washing with soap. However, KII results show that many host communities had inadequate water supply. To promote sustained behaviour of hand-washing with soap and effective use of toilets, availability of water at both communities and schools is necessary. FGD results and site observations in six communities visited show that existing water supply systems both in communities and schools were inadequate as shown by broken facilities or not enough water in the dry season. Similarly the HHS results show that across three districts surveyed (details presented in the effectiveness section). An increase in number of water supply systems construction from 3 to 8 by June 2015 means Plan has incorporated the recommendation from the WASH evaluation By increasing the number of CWS, it has addressed the need for water supply among the target communities as it emerged. As staff has pointed out the assessment of the water facilities at the beginning were not accurate. The assessment did not assess whether the facilities were functioning well or not. Therefore when the implementation stated to progress it had been found that many water supplies were not functioning. Thus Plan has to increase construction of rehabilitation of CWS. It can be concluded that construction of water supply systems is very relevant to the community s needs. Furthermore, the majority of the officials and plan staff reported that they had not done enough to provide water supply to communities. They acknowledged that this comes down to the limited funding available for infrastructure construction. The KIIs revealed that initial assessment of the water systems was inaccurate saying broken facilities were not reported properly. 20 Water supplies (ideally) in villages and schools are functioning adequately to meet village needs. If not, adequate resources either from Plan, government or other organizations can be mobilized during program implementation process; 37

57 Sanitation component WASH Program adopted CLTS to pursue its objective of ODF in the target communities. Adoption of this approach has been endorsed by the government through its recognition of the approach in National Plan of Action for RWSS. KII results confirmed that direct sanitation subsidies had not been working meaning low rate of effective use and poor maintenance. Key informants reported to the evaluators that there were multiple reasons for villagers not using toilet and not maintaining it well. One of the reasons is that there was inadequate water, broken facilities or lack of water during the dry seasons. In terms of toilet use, some said that people don t have habits of using it or there are still bushes around so that it is more convenient for them to go defecate in the open. The evaluation team found that Plan s staff has internalised the CLTS approach well. The government counterparts also believed the approach has been successful in encouraging households to take actions towards sanitation improvement. However, government counterparts also admitted that CLTS results occur very slowly which sometimes do not match the urgent need to meet government (as well as Plan) targets in time. The team also found that there were many CLTS target villages that had been introduced with the government funded subsidies (i.e. Mokso village in Pha Oudom district). The PHD has been inconsistent in their view on CLTS. In 2013, when a pro-poor sanitation strategy using smart subsidies was considered they opposed it. Now, they introduced a none-discriminatory subsidies to a group of chosen villages (pro-poor not being considered). It is not a planned activity to introduce subsidy in the target villages or at least not in the Plan s plan. However, the WASH pilot project evaluation in 2012 suggested that Plan study the potential use of smart subsidies. A smart subsides review was undertaken in Given the complexities of smart subsides, Plan Laos decided not to pursue smart subsides within the project until sanitation marketing options had been tested first within the target areas. A constraint on the activities of the TWG is that it lacks a staffed secretariat or at least a fulltime officer to spearhead its work beyond the monthly meetings, for example working on advocacy and resource mobilization for sanitation. Currently there is only a designated coordinator for whom this is not a full-time job. Hygiene promotion component This component aims at behavioral changes in hygiene practices at school and community level. This component is well aligned with the education sector strategy regarding improving hygiene practices amongst children. It is clear that some indicators within this component are aligned with the SoQ indicators. Also some indicators are aligned with MOH indicators concerning MHV. Therefore, this component is aligned with the sectoral strategy. The FGDs results indicate that hygiene awareness-raising is very relevant particularly among children. Children participating in the FGDS reported that washing hand with soap has become a new practice. A place to wash hands, however, seems to be a new concept and difficult to buy in. The evaluators explored the concept through FGDs and site observation and came to conclusion that a place to wash hands for the villagers means a bucket with water and soap (dish washing liquid included). The common practice for hand washing takes place in the morning when they wake up, in the evening when they take shower (or wash their body). This is the time when they wash hands with soap. Before and after meals is also common time for hand washing without soap. For those who have toilets, after using toilet is 38

58 also a time for hand washing. Awareness raising in relation to washing hand with soap is very relevant to improve hygiene practice and need more efforts (more analyse will be provided in the effectiveness section). The majority of FGDs participants including children groups were well aware of the importance of drinking water treated and stored properly. The participants acknowledged that drinking un-boiled water is a major cause of diarrhoea and other diseases. When probed whether everyone drinks boiled water all the time, the participants admitted most villages drink boiled water when they are at home which means they drink un-boiled water when they are out working in the field. The evaluators did not have a site inspection at schools, but through interviews with VEDCs we learned that most school students drink treated water by using the project distributed water filters and storing it in a closed container. 3.2 Relevance of the MNCH-N Program The MNCH-N Program in Bokeo is well aligned with the government policy and relevant to the current needs of the province. The project design was consistent with the need of the GoL to provide a universal access to health care services particularly for remote rural population with an aim to reduce malnutrition and child and maternal mortality rates. The program activities complemented the MNCH-N strategies of the government. The program also met the need of the community. This is because the Program design was based on the participatory stakeholder consultation and need assessment. The program remained relevant throughout the program period. The current HHS shows that the number of women delivered at health facility remains low (27.0%). The majority of women surveyed delivered at home (66.7%) and received assistance from relatives/friends (62.2%). The ANC visit 4 times is 47.2%. However, the newborn and child health care show high indication with 70.8% of newborn were protected against tetanus and 74.8% of new born was breastfeed within one hour. It is evident that the MNCH-N Program in Bokeo had been designed to implement through the existing health care system which makes it coherent with the government structure. The target areas were selected appropriately. As the 2011 LSIS indicated, in Bokeo 46% of children under five years were stunted which is above the national figure (44%) and also infant and under five mortality rates are above the national figure (92/1000 LB and 110/1000LB vs 68/1000LB and 73/1000LB, respectively). The three target districts were the poorest of the province. Therefore, targeting from selection of the province to district prioritisation was very appropriate. The program responds to the community and local government institution needs. During the implementation of the Program, government institutions like Nutrition Centre, MCH Centre, and EPI were invited by Plan to be core lecturers so that it reflected the common understanding concerning to MNCH-N lessons and ensured the implementation goes in the same direction. It had achieved the contextual harmonization of working groups between project staffs and government partners. This continued engagement with the government counterpart also ensured the Program remained relevant and coherent with the government plan and approach. 39

59 Delivery mechanism A steering committee for the project was set up at the provincial level. Following the evaluation in 2012, which reported the less than satisfactory experience of Plan and other organisations with provincial level committees and implementation the function of the provincial committee, was shifted to the district level. This means a reduced level of provincial involvement. The KII results show that the distribution of roles and responsibilities between the district and provincial authorities has caused a certain level of difficulty. Cooperation at a higher level is ineffective resulting in a lack of cohesion of the WASH and MNCH-N intervention. For example, the KII respondents reported that evaluation of MHV conditions was undertaken without an involvement of WASH staff (the issue was internal though). The issue of ineffective communication between district and provincial authorities was also raised. For example, the districts authorities reported to their provincial counterparts in a written form, but the provincial authorities demanded an oral reporting as well. The higher level was not updated on the progress of the project by the district as it was supposed to be the case. DWTS and CLTS Committee were established as delivery mechanism for the WASH activities. The KII results show also that there was lack compliance with their roles by DWTS and CLTS Committee members. Members of the DWTs reported during the interviews that some of the members did go to the field but were not very active particularly those who are not from health background. Monitoring and reporting The WASH Program has adopted a comprehensive monitoring and reporting system. The monitoring system disaggregated data by indicators, genders, poverty and social inclusion indicators such as disability inclusiveness. As recommended by the 2012 evaluation, some indicators were harmonised with those of the MNCH-N Program. It was suggested that Plan Harmonize M&E indicators with the School of Quality and Model Healthy Village indicators of government while internal indicators relevant to Plan long-term visions such as ODF to be maintained. To ensure coverage of both government indicators, it was also suggested that the WASH, Education and Health Programs of Plan have a common/ integrated M&E system. Focus on nutrition Nutrition is now a priority on the Ministry of Health agenda and with WASH, Mother and Child Health and Nutrition co-located under the same Directorate there is real potential to better plan, coordinate and develop mutually supportive interventions, including both nutrition specific and nutrition-sensitive interventions such as WASH. Members of Nam Saat are now regular attendees of meetings of the Nutrition Technical Working Group. National learning events hosted by Nam Saat, and supported by [Plan], the World Bank, have on several occasions invited representatives from the national Nutrition Centre to discuss the interrelationships and operational linkages between nutrition and WASH 21. The FNSAP was developed in the second half of 2013 in response to a growing awareness that the country was off track to meet its nutrition related targets under MDG 1. This was confirmed by the 2013 MDG Report which highlighted an increase in food poverty as well as 21 Lao People's Democratic Republic Strengthening Water Supply, Sanitation and Hygiene Sector Coordination in Lao PDR: Supporting Sector Reform for Scaling Up Rural Sanitation Synthesis Report 40

60 high rates of malnutrition and stunted growth among children12. In mid-2013, the Prime Minister signed a decree establishing the National Nutrition Committee and Secretariat (now based within the Ministry of Health), and the Multi-sectoral Food and Nutrition Security Action Plan (FNSAP) was finalised with responsibility for coordination placed with the National Nutrition Committee. Since late 2013, there has been a commitment to improving coordination and alignment by both Government as well as Development Partners, in addressing chronic malnutrition. A 'road map' for implementation was developed and agreement reached on starting implementation in six districts in three provinces - Luang Namtha, Saravane and Oudomxay. At the time of writing this report, micro-planning had commenced in Oudomxay Province, including the education, health and agricultural sectors. The renewed focus on nutrition is also in line with the MOH s effort to improve the nutrition status of children under-five years across the country with the approach called Convergence Plan. This approached is well adopted by DPs such as UNICEF, WHO, Save the Children and WFP. The plan involves relevant sectors including health, education, WASH and agriculture and forestry. 41

61 4 Effectiveness 4.1 Effectiveness of the WASH program This section presents findings and analysis of four major components of the WASH project including hygiene promotion, SLTS/CLTS, construction/rehabilitation of WASH facilities and WASH sector governance. Hygiene promotion component The interventions under this component mainly included hygiene promotion activities undertaken by DWTs on a regular basis in villages and schools, hygiene promotion events to reinforce hygiene promotion messages undertaken by DWTs at the target communities and schools and support monitoring of the exiting government program such as MHV and SoQ. Overall, the evaluation found that the project s hygiene promotion activities were effective in raising awareness and increasing knowledge about improved hygiene practices. The project had variable results in turning this hygiene knowledge to practice. The evaluation team managed to conduct FGDs with students in the communities outside schools setting. The students knew least three critical times to wash their hands. They reported that they applied hand washing steps every day. When asked if it has a knock on effect on their parents, the responses were negative. They said their parents were not interested in washing hand in many steps. This suggested that that the project s strategy of school children taking messages home to influence the rest of their family s hygiene behaviors was variably successful. Students both boy and girls groups reported that they drink water treated and stored properly. However we also observed that some children drank water directly from the taps (public water points) suggesting that this knowledge has not necessarily been turned into practice. In some villages, students also reported that they have water filter devices. Some said their teachers do not know how to use the filter properly. We crossed checked with HC staff if it would be possible if a teacher would not know how to use the filters. HC staff confirmed that that could be the case. Even though the teachers or at least the school principals were oriented about how to the device, in some schools those oriented left the school without proper transition of tasks and routines. Improved HP at HH level is reported to have been adopted by villagers in target communities. FGD respondents reported that they did not have a concept of washing hands with soap before the project started. Now they learned that they should wash hands with soap at critical times (such as after defecation, after cleaning children s bottoms, before eating, before feeding children and before preparing food).. However, they also admitted that washing hands with soap is not a common practice particularly before eating. For them, washing hands with soap is done in the morning when they wash their face or take a shower and evening when they return from the field. They also reported that hand washing (without soaps) takes place before and after meals (usually three meals a day). It was observed that a place to wash hands was available in the form of a bucket of water which is usually for general purpose and not for handwashing only. A sink, tippy tap or other type of handwashing place was not present. The project did not promote that a flowing water source when washing hands is more hygienic than within a closed container (e.g. bucket) which can be a source of recontamination. As seen in Figure 4 below the rate of respondents 42

62 who have a place to wash hands varies significantly from 94% in Meung to just 37 % in Pha oudom. After cross-checking with FGDs and interviews, this could mean the respondents interpreted the concept of a place to wash hands differently. FGD and interview respondents reported that a place to wash hands as a sink does not exist but a bucket of water is available at almost all households. If the respondents replied that they did not have a place to wash hands they might not have a sink. Even though the survey team agreed that a place to wash hands is a designated place for washing hands where water and soap are present all the time. This can be a weakness in administration of the survey, but also this tells us that the concept is open for interpretation. Figure 4: A place to wash hand and hand washing with soap Source: this evaluation HHS, 2015 Another important finding is the rate of respondents who know at least three critical times to wash hands (target 80%). As Figure 4 below shows, only about 19% of the respondents know at least three critical times to wash their hand which is significantly below the target of 80%. In all districts, the results of 10%, 17% and 29% were lower than the 2013 baseline (Meung 23%, Paktha 12% Pha Oudom 35%). The different sampling technique adopted may explain the results. In the baseline survey, more women and easier accessible communities were sampled. Thus, the results in relation to knowledge about the critical time to wash hands can be higher this evaluation. FGD results show variation opinions amongst men and women with women seemed more knowledgeable about critical time to wash hand. A number of reasons could be attributed to this including the effectiveness of the hygiene promotion sessions. Another reason could be that some community members also received hygiene knowledge from health education delivered through mobile health outreaches. The HC staff interviewed reported that more women were engaged during the outreaches. This suggests that reinforcing key hygiene messages from different sources is a useful strategy, but may also mean that CLTS committees at the community level were less effective in promoting hand washing with soap and motivating community members to change their hygiene behaviour habitually. KII respondents confirmed that CLTS committee members were less effective in encouraging 43

63 other villagers to improve health and hygiene conditions because they are also villagers. Some of them are shy and not skilful in explaining things. Figure 5: Knowledge about critical time to wash hand Source: this evaluation HHS, 2015 Knowledge about the causes of diarrhea can be associated with the motivation for villagers to decide to build a toilet. All FGDs respondents including boy and girl groups reported that diarrhea is caused mainly by open defecation and drinking of untreated water (un-boiled water). They also said that un-boiled water is contaminated because they practice open defecation. HHS results show that eating and drinking cause diarrhea. With the majority reported eating food without properly cooked cause diarrhea. See Figure 6 below. Figure 6: Knowledge about causes of diarrhea Source: this evaluation HHS,

64 This result could have been attained through many interventions by Plan such as PPGs, MNCH-N and WASH which covered improved sanitation and hygiene practices. It is not reasonable to disaggregate HHS data by program-based interventions. From FGDs, we found no major difference in the results related behavioral changes between men and women. i.e. improved hygiene practices. The table below show the number of people who had participated in the SLTS/CLTS sessions. By June 2015, the total number of people who had participated in the hygiene promotion activities reached 15,611 including men, women, boys and girls. It can be seen clearly that the project had promoted participation of women and children resulting in 576 more women than men participated in the sessions. Table 13: Number of people with increase knowledge of hygiene practices Men Women Boys Girls TOTAL 5,835 6,411 2,011 1,354 15,611 Source: WASH annual report FY15 CLTS Key interventions under this component included establishment of DWTs in each target district, community and school consultation/orientation including establishment of CLTS committees and water committees, construction of WASH facilities in schools and communities, CLTS triggering and follow ups, and technical support of operation and maintenance of WASH facilities. This evaluation confirmed the effectiveness of the CLTS approach. Figure 9 below shows a number of respondents participated in the triggering sessions against the number of new toilet constructed in the year. The table shows the effectiveness of CLTS in Meung district where in 2012 a higher number of respondents reported to have participated in a triggering session corresponds with a higher number of toilets being constructed. The table below shows clearly that the triggering and follow up sessions were effective in encouraging people to build toilets. The follow-up sessions/hygiene promotion was primarily focused on stopping open defection, and hand washing with soap was seen as a secondary action. Given that other conditions are more or less the same because they were randomly sampled for the survey, higher proportion of people participated in the CLTS sessions build toilets. 45

65 Table 14: Number of people participated in the CLTS sessions Meung Paktha Pha-oudom No % No % No % Participated* Have toilet now 35 71% 71 73% 85 73% Not participated* Have toilet now 33 50% 9 50% 11 34% *including participation in the follow up sessions Source: this evaluation HHS, 2015 It is important to note that even though the CLTS approach is effective at ceasing open defection in villages, the toilet coverage rate is still low because implementation of the approach was not done properly. These include the fact that the teams could ensure triggiering and follow-up sessions were undertaken at a time where maximum participation by community member (or a representative of a HH) were able to attend. Further, the project didn t make concerted efforts to undertake additional triggering sessions for villages that were not able to attend the original session. The table below shows survey respondents participation in the CLTS sessions. In Meung in particular, there were fewer respondents participated in the sessions than those who were not. Across three districts, a high proportion of respondents participated in the CLTS sessions eventually built their toilets (71 % in Meng, 73 % in Paktha and 73 % in Pha Oudom. See table below for more details). Table 15: CLTS participation and construction of the toilets Meung Paktha Pha-oudom No % No % No % Participation in the CLTS sessions 49 43% 97 84% % No Participation in the CLTS sessions 66 57% 18 16% 32 22% Source: this evaluation HHS, 2015 There are challenges contributing to effectiveness of CLTS. Firstly, there was difficulty operatinalising DWTs. Planning for visits to target communities involves a lot of team building and logistic preparation. This was reported to be quite a challenging exercise. As mentioned earlier, the team is multi-sectoral including health, education, LWU and LYU. In the interviews, DWT members reported that it was quite a challenge to form a full team for CLTS community visits and more so lately when there were members from District Rural Development Office and Planning Office who joined the team. They claimed some DWT members particularly those from District Education Offices were in charge of too many responsibilities and the project s responsibilities were seen outside their core mandate. The reason was the effective communication and lack of a team long-term plan. The difference between ethnic groups is explored in the table below. On average, about 50 % of the villagers surveyed participated in CLTS process (triggering session which takes place once in a village). The lower the number of villagers who showed up in these sessions, the more difficult it would be for villagers to decide to build a toilet. In other words, if they are 46

66 not triggered there would be no desire to change Figure 11 below also shows that the rate of participation varies from ethnic to ethnic with Kouy and Aka among the lower rate of participation and Muser is the lowest. In consequence, the rate of respondents having a toilet also varies with Muser having the least number of toilets. Figure 7: CLTS participation and access to sanitation by ethnic groups surveyed Source: this evaluation HHS, 2015 The DWT members interviewed revealed that with additional team members it did not mean that the team would become more effective. Some members were just hanging around and did not perform any important team tasks. The reasons included those hanging around lack technical background in primary health care; the trainings were not frequent enough, there was no clear leadership structure within the team to facilitate team work. The evaluation team found that there were other conditions contributing to household investment in the toilet construction or effectiveness of CLTS. These include subsidies from the government or a history of subsidies by other projects, availability of employment which increases the financial capacity of the villagers. Many key informant interview participants expressed their view against CLTS approach claiming it was not effective in the Lao context for reasons such as there were still a lot of forest nearby the villages, and water supply was not reliable these views demonstrate that behaviour change interventions had not been fully successful. While the CLTS approach might not lead to ODF in a short timeframe it can lead to increase in toilet coverage and effective use. The project had 3 years in a pilot phase which was followed by 4 years in full implementation so sanitation progress was a lot lower than anticipated. However, this is within a context where the project has not been operating in a strong enabling environment for CLTS and hence the project had to make concerted efforts to improve this simultaneously. It is important to highlight that the HH survey findings show a high proportion of respondents did not participate in the CLTS triggering session. This perhaps is part of the reason why people believed CLTS was not effective. Table 8 shows numbers and proportion of respondents participation in the CLTS triggering session. At the community level, the CLTS process faced some advantages and challenges. DWT members reported that interns that Plan has supported helped a lot in terms building trust 47

67 between outsiders and the community members. The trust between them was important because it made people come to the meetings, without it people would not come to the sessions thus would not receive messages. The CLTS process were also interactive and engaged. People were having fun while absorbing the messages. However, there were challenged DWTs across three districts shared. They all had a few difficult villages. This means that people were not cooperative, they did not come to the meetings, they usually were the ethnics that speak limited Lao, and often spend most of the time out in the fields. Thus they faced the challenges in terms of language barriers, deeply rooted habits of poor hygiene practices amongst the ethnic minority groups and limited number of people joining the sessions. The project did not have specific and nuanced strategies to work with different ethnic groups which were not only a missed opportunity but a necessity when working with ethnic diverse communities. It is less evident in the assumption that hhouseholds and communities upgrade to more durable toilets types over the long-term. KII respondents reported that poverty is the key reason. They said CLTS and hygiene promotion activities were effective among better off households, but less effective for the poor. Even these people were aware of the important of the toilet and wanted to upgrade it they still could not due to their financial barrier. There is also limited implication from the sanitation marketing exercise. This is limited demand for sanitation facilities that encourage private sector to expand goods and services to remote areas. The Evaluation Team did not meet with any sanitation suppliers. So we were not able to interview any suppliers. Monitoring on WASH activities and practices in schools and villages being adopted and sustained by relevant government provincial and district agencies is unlikely if there is no project support. At the time of the evaluation, the provincial and district authorities reported they won t have budget specifically allocated for WASH activities in general. Maintenance of water facilities in schools has not been integrated to standard activities of relevant government agencies. It has to be noted that Plan s supported follow ups of WASH facility only last for one year at a particular community. The facilities were mainly built one or one and half years ago when the team visited. The VECs were not very clear about how they would find the money for WASH in schools maintenance. Construction/rehabilitation of WASH facilities Construction/rehabilitation of WASH facilities has contributed to the effectiveness of HP in schools as the facilities (where functional) provide beneficiaries in schools with the means to adopt and practice the recommended sanitation and hygiene practices. Tools and approaches used in promoting hygiene and sanitation such as the Bluebox, the SSHT toolkit, and the CLTS toolkit all recognized and accepted tools by the Government and the puppet shows from the project appear to be effective means to convey messages to beneficiaries, particularly children. The challenge remains at the level of behavioral change. KII respondents strongly claimed that some ethnics groups did join the puppet shows, events and health education sessions but they did not put into actions. As stated above, this could be because specific and nuanced strategies to work with different ethnic groups was not applied within the project. In terms of toilet coverage, the project has made significant progress toward to the 2016 target. As in Figure 12 below, there is only 3 % below the target of 80 % for toilet overage across the target communities. Pha Oudom has seen lowest increase in the coverage as the 48

68 WASH did not expanded to many of the villages for the imminent resettlement because of the dam construction along the Tha river. Figure 8: Sanitation coverage in target district against baseline Source: WASH annual report FY15, Plan Laos. Achievement of ODF status remains a challenge. Overall, just above 50 % of the 2016 target had been achieved (see Figure 5 below). In particular, Meung had made the least performance compared to other two districts (48% for village and 50% for school). KII respondents reported that achieving ODF status was ambitious for the following reasons. First of all, the timeframe of the project in their opinion was short and the impact of CLTS was slow. They argued that ensuring elimination of OD would not be easy: deeply rooted habits of OD, financial constraints and availability of water supply. Another reason was associated with expansion of the community. This means that it was common to see young families, newly married couple, to separate from their parents house and build new one. During the first few years they were unlikely to be able to afford the toilet. When asked if they could share with their parents, the respondents said that could not happen all the time since their new house was not necessary close to their parents. Other reasons included resettlements and disaster. In terms of resettlement, it was reported that new comers often settled adjacent to the existing communities and they could not benefit from the existing water supply system, without water it was very difficult for them to build and use toilets. Figure 9: ODF achievement in villages and schools Source: WASH annual report FY15, Plan Laos. 49

69 Participatory governance in WASH sector In general, the WASH sector governance in Bokeo has undergone major transformations since Plan s support has definitely contributed to these positive changes for the governance of the sector at provincial, district and community levels. Establishment of DWTs was one of its kind even exist in the province. The same for CLTS committees and having in place WSPs. The governance structure at the same time promoted gender equality and social inclusion. It promoted gender equality by encouraging more women to assume positions in the committees including in the DWTs. As presented in the previous section, 50-60% of the DWT members were women. There were women in the CLTS committees and 14% assumed leadership positions. Gender quality was also promoted at HH level. However, the role of women in decision making about toilet construction remained minimal. As shown in Figure 7 below, there were very few women or wife made the decision but the majority of respondents reported they made the decision together. There is no baseline information to compare the progress on this topic, but in relation to governance it has been the intervention that promotes gender equality. Figure 10: the role of men and women in decision making about toilet construction Decision making concerning investment in toilet construction also varies from one ethnic to another. Joint decision making means husband and wife make decision together. The Figure shows that Laoloum demonstrate higher percentage of household jointly made decisionmore gender equality regarding toilet construction. Higher inequality can be found in Aka and Hmong. This will be discussed more in the gender, accountability and social inclusion section. For all ethnic groups, the graph below shows that hardly any women made the final decision with regards to building a toilet at their household. However, they were able to influence the decision whether the HH will build a toilet or not in the joint decision mode. A joint decision making means either husband or wife can veto the decision if one disagrees. Working relationships with government partners The Project established and maintains working relationships with the NamSaat department at the national level, with the PHD and PED at the provincial level, and with DWTs and the District Governor at the district level. With non-government organizations and donor 50

70 partners, the Project actively participates in the TWG, providing technical inputs to proposed policy or approaches in the WASH sector, and also supports the regular meetings and reporting of the group. At central level, the KII respondents from the government counterparts shared the view that Plan should engaged with line Ministries from central down to local levels. This may not reflect the government point of view as it has been agreed that such development project as Plan s ones can coordinate with the Provincial level. It is clear in the MoUs that many of the reporting line i.e. from district Good working relationships with the government also depend on the relationships between project staff and that of government formal and informal. It was reported through KIIs that at national level, the Program Manager has worked well with the NamSaat 22 while the Provincial WASH Coordinator has good working relationships with the provincial agency and district agency staffs. Project district WASH staff also has to have good working relationships with their counterparts in the different agencies at district level as they work closely and often. A formal agreement between the Project and government would greatly clarify the roles and responsibilities of both parties, including those of the WASH officers and of the DWTs. During the interviews, Plan staff described their working relationship with the government partners as good working relations. They were quick to raised issues of the official availability and commitments to other responsibility that affect the implementation of the Program activities. Plan staff shared the views that assigned government staff take the responsibilities to Plan supported development activities secondarily compared to other priorities. Some claimed that the assigned officials had not been acknowledged for their performance working with INGO supported initiatives by their office recognition system. Therefore, they have not been too active working with Plan. Some district staff still don t understanding the ownership of the development activities. It was supposed that plan and reports be shared at the district level. It is the district responsibility to share the report and plans to their higher levels. The same should apply for the PHD with the central level. Ownership of the district staff has not been informed by their leadership management. Coordination between plan and government authority need better understanding. On the other hand, the government counterparts had also something to say about Plan staff. They claimed Plan staff didn t coordinate well in rolling out activities particularly during the planning process. They said lack of effective communication make it difficult for them to integrate Plan activities into their schedule. At the provincial level, PHO said in worse case, short notices often came from Plan which forced them to allocate staff who might not be those trained and cooperated with Plan before. This affects the effectiveness of what they are doing. 22 Consultant s perception and observations from several discussions with the Program Manager. Unfortunately, the Consultant had no opportunity to meet with other WASH NGOs or other organizations working with NamSaat at the national level due to time constraints. The Consultant requested an interview with the NamSaat representative who participated in the Design Workshop but was reluctant to discuss these issues without explicit permission from the NamSaat Director.. 51

71 These accounts are not novel. They have been documents in other reviews. However, it is worth repeating these issues again for the following reasons. First is that they implies dissatisfaction from both sides over one another. This can negatively affect the implementation of the project activities in the future and it can get worse if left unaddressed. The second reason is that the issues raised by the two sides are interrelated: officials availability and lack of proper coordination. This means that to some extent they were telling the true. 4.2 Effectiveness of the MNCH-N Program The evaluation found that there are seven underlying factors that could influence the achievements in the MNCH-N Program in Bokeo. For health providers: 1) All year round access to villages is important. Difficult road condition in the rainy season, mountainous areas, scattering of settlement and resettlement made the implementation of some health events delayed; and 2) There were unclear roles and responsibilities amongst district health staff and health centre staff for the conduction of health events in communities particularly as for who would take lead in conducting health events. For communities: 3) Percentage of women who preferred delivery at home was high. 66.7% of women surveyed reported to delivery at home and most delivery was assisted by relatives/friends (62.2%). Cultural beliefs and distance were the key factors; 4) Limited knowledge on child feeding practices and shortage of food for households because communities mainly rely on food available in the forest including vegetables, mushrooms, bamboo shoots, fish and etc. The HHS shows that the percentage of children 6-23 months received exclusive breastfeeding for the first six months is low and even lower than the baseline and national figures (33.8% vs 52.7% vs 40.4%, respectively). The percentage of children 6-23 months received four or more food groups is low in both end line and baseline surveys and lower than the national figure (20.9% vs 20.6% vs 35.0%, respectively). In addition, FGD participants expressed that if the child does not eat properly it is normal for them; 5) Distance from home to health facility plays a role. The survey shows that some women reported to live far from health facilities, over 5 km or more than one hour walk (39.2%). Even though the internal review found that 60% of the HHs own a motorbike and it could be a good mode of transport for ANC and other health problems, it is not suitable for delivery or final stage of pregnant check-ups; 6) Low education attainment, 42.7% of respondents are without schooling and 31.2% attended only primary school. Low education causes poor health seeking behaviours as FGD participants accepted; and 7) Language barrier most women cannot speak Lao language so it would be difficult when seeking health care and communicating for behavioral changes (cited by field KII respondents), this survey is also evident that 90.4% of respondents are ethnic people from 52

72 Muser, Hmong, Koui, Khmu, Lamade, and AkHHa. Therefore for the next MNCH-N intervention the seven core important factors should be considered carefully. Access to MNCH-N services The program had successfully achieved the objective to improve the access to health care services in the target communities thus contributing to the Bokeo province to achieve the MDG 4 and 5. As the HHS indicates, access to ANC visit 4 times, PNC visit within 7 days of delivery, PNC visit within 6 weeks and PNC visit 3 times had increased over time and was particularly so comparing to the baseline survey. In addition, more WRA had an increased access to family planning services, tetanus toxoid vaccination and iron supplementation. Mothers have had better knowledge on diseases that drive children to health facility immediately (Figure 10). This achievement is consistent with the results from the desk review concerning the number beneficiaries learned about health education from health events conducted during the project life. Moreover, access to new born and child health care had improved over the time. After the program intervention more and more children had access to child health care services except for children 6-23 months access to food 4 food groups which remain unchanged (Figure 11). It is true that access to more food groups remains the same compared to the baseline. The FGD results revealed that villagers mainly rely on food available in the nature such as forest vegetables, mushroom, bamboo, game and fish in rivers. While these are nutritious food but the quantity and diversity and quality are very low thus this dietary style minimally contributed to improvement of the nutrition status. In addition, the findings show that mothers have better knowledge on four reasons that needs to take a child to health facility (7.7% in this survey whereas 3.4% was in the baseline survey. This is because of the program provides the parenting orientation regarding MNCH-N in the communities by trained DHO /VHV and village facilitators and this implementation has been monthly monitored. Figure 11: The access to mother and child health services at baseline survey compared to the end line survey in

73 Regarding delivery, the program had not achieved the target to encourage mothers to give birth at health facilities and assisted by SBA (Table 16). The key informants explained that it was hard to change this behaviour within a short time frame because this culture was deeply rooted in most Lao mothers. They were programed to give birth at home particularly among the ethnic women. FGD results also revealed that they were shame to give birth with other people presence rather than family members or relatives. The KII respondents particularly the HC staff reported that ethnic women in the target communities particularly Hmong were the least group using the health facility for child delivery. They often sought help when they had complication i.e. prolong laboring, excessive bleeding and the like. Even so they would make a phone call to check if the female health staff were available at the facilities. This means that they are too shy to allow male staff to provide service even when their life was at great risks. Figure 12: The access to new born and child care at baseline survey compared to the end line survey in 2015 The continuation of health education conducted during the outreach activities and implemented as part of parenting orientation in communities had contributed to behavioural changes amongst the beneficiaries. However, birth delivery had not significantly changed for the following reasons. FGD participants with men and women said that health education usually participated by women, but the decision whether they would use health facilities for delivery was made mainly by men. Women also did not want other people to see their secret part of the body. Men also did not want other to see that part of their wife. Many were still confused about free delivery policy including HC staff. Some claimed they were charged when using health facilities for child delivery. Transportation and distance to the facilities were also mentioned as part of the reason. Table 16 below should be viewed in comparison to the baseline as followings. The current Lao Social Indicator Survey (LSIS), completed in 2011, reported that PW had attended ANC visit 4 times was 36.9%, delivered with skill birth attendants was 41.5%, and delivered at health facility was 37.5%. After delivery 39.5% of PNC women visited PNC services and 40.6% 54

74 brought newborn for health checked. In addition, 18.2% of women got pregnant at early age and 49.8% of women in reproductive aged had utilized the contraceptive 23. Table 16: Status of delivery at baseline and end-line survey Item Baseline survey End line survey National figure Delivery at health facility 29.6% 27.0% 37.5% Delivery at home na 66.7% Na Delivery assisted by SBA 31.5% 29.7% 41.5% Delivery assisted by relative and friends na 62.2% na 24 Effectiveness of knowledge of mothers on the danger signs of pregnancy and delivery The knowledge of mothers on danger signs of pregnancy and delivery had been improved. The survey results show that more mothers were aware of at least two and 3 critical signs of PNC that they need to seek care at health facilities (39.6% and 18.9%, respectively). This improvement was corresponding to an increase of access to PNC service in this HHS such as the access to PNC within 7 days and within 6 weeks and the number of PNC visit 3 times has been increased over time during the project life. Similar observation is noted for the mothers knowledge on critical signs of labor course that need to seek care at health facilities (18%). This positive change of mothers knowledge is a consequence from the parenting orientation activities implemented in communities that included various topics related to MNCH-N. The parenting orientation activities are effective in increasing the knowledge of mothers and influenced health seeking behaviours regarding delivery and PNC service with SBA or at health facilities. These achievements can attributed to the program design that addresses the local needs. The program mainly focuses on the provision of quality health services in the target communities and to reach all people in remote areas. The SBAs education significantly complemented the MNCH-N activities while addressing the cultural and language obstacles. For instance, the ethnic women were too shy to utilize health care services with male health staff delivering the ANC visit. Strengthening of the capacity of the health staff does lead to better management of outreach activities and launching of health events in the communities. The outreach activities are very appropriate for the rural remote setting regarding access to health care services. The KII respondents in Bokeo stated that poverty and low education are the reasons to blame for villagers not seeking health care services. The limited exposure to modern health treatment is also another reason. Most mother perceived that when children are sick or crying and do not eat properly, they thought it is common for their children. Sometime they thought that these are part of the growing up of the child. 23 Lao Social Indicator Survey (LSIS) December MOH, NSC, UNFPA and UNICEF. 24 Please note that the sampling technique adopted in baseline survey and in this evaluation is different therefore we are not making a direct comparison. It should also be note that the coverage rate of the intervention is almost 100% of the population so the sampling technique would not make much difference in terms of the probability to be sampled of the population. 55

75 Effectiveness on nutrition status of children Concerning to the nutrition status of children < 2 years, the evaluation observes that the program had contributed to improving the nutrition status of the target children particularly wasting (weight for height/length) among children < 2 years of age. Figure 12 below shows that percentage of wasting had decreased, severe and moderate wasting rates had decreased from 6.5% and 13.7% as reported in the baseline to 4.3% and 6.5% respectively. This improvement is a synergy effect of the integration between WASH and MNCH-N programs. The HHS shows that hygiene and sanitation in target villages has been improved, 78% of target villages achieved ODF, 34.5% of mothers had better knowledge on how to dispose of child feces properly, at least people knew about three critical times to wash their hand (Meung 23%, Paktha 12% and Pha Oudom 35%). FGD participants show that they knew the cause of diarrhoea correctly. This improvement could contribute to reducing the diarrhoea episode among children U5Y, but in this survey we did not measure the diarrhoea episode, and consequently improved nutrition status of children. Figure 13: Wasting (Weight for Height/length) by district In addition, the MNCH-N program activities was successful in encouraging mothers to feed baby with colostrum within one hour of birth (74.8%), and to give complementary food beside the breastfeeding to children 6-9 months (92.0%). The percentage of wasting in this HHS is still higher than the national figure (severe wasting is 1.4% and moderate wasting is 5.9%, LSIS). In order to improve the wasting in the target communities in the future Plan needs to focus on the nutrition status of children aged between 0-5 months because they are the most suffered from both severe and moderate wasting (Figure 13). The promotion of exclusive breastfeeding for the first six months of age should be emphasized intensively in order to ensuring the child growth. In this HHS, the exclusive breastfeeding is lower than the baseline (this HHS 33.8% vs baseline survey 52.7%). It is important to note that children aged 0 5 months had moderate wasting whereas children aged 6 23 months were moderately stunted when compared with other age groups (Figure 13). This finding is consistent with the results from the interviews with mothers who accepted that they were not able to provide their children with a variety quality food groups. Few children received a minimum acceptable diet apart from milk, breastfeeding (Figure 11). In Lao PDR, most mothers introduced grilled masticated sticky rice wrapped by banana leave to child at early age and some mothers introduced it even few days after birth. 25 In-depth interviews with local authorities and observation in villages also indicate that mothers have 25 NIOPH and UNICEF, Formative research on young child feeding practices in southern Lao PDR. 56

76 low education, ethnic people and cannot speak Lao language so that the access to media for health education on nutrition is very limited. Household food insecurity is common in villages because it mainly rely on the natural sources or forest surrounding villages such as vegetables, fish, games, bamboo, mushroom etc. but forest food is scare in dry season (the dry season takes about 6 months of the year). Moreover, mothers have poor knowledge on young child feeding practices. Therefore, the factors contributing to wasting and stunting among children are multi-facets and need multi-sectoral solutions. To improve food availability and accessibility, an integration of MNCH-N, WASH and agriculture programming could be part of the solutions. While MNCH-N and WASH improved hygiene practice and increase knowledge on proper dietary, agriculture will promote availability of food. For example, agriculture can introduce household gardening, food demonstration that uses locally available or home grown food. The KII results and field observation all came to a conclusion that food availability and accessibility remain a major challenge for the target communities. They claimed that introducing agriculture component into the program would greatly contribute to increase access to food. During the field work in villages the evaluators noted that mothers did not pay attention to child s health even the child had mild fever and were crying and mother still said that it is normal for the child to have such conditions. The MNCH-N and WASH programs started around the same time and there were some important evidence to show that the two programs were complementing each other. For example, health facilities including 11 health centers and 02 DHO had access to safe and clean water and clean toilets which belong to activities under WASH. In addition, many villages declared ODF and mothers dispose of child feces in a proper way as a result of the two programs promoting convergent health messages. Therefore, for instance, diarrhea cases among children had been decreased (one of the main cause for malnutrition among children. moreover, many villages achieved MHV; a high number of target beneficiaries accessed MNCH-N services: ANC and PNC visits in communities, etc. this evidence points to the conclusion that multi-sectoral program should be paid more attention to. Figure 14: Wasting by age in months and district in target districts 57

77 The HHS indicates that the program has not achieved to reducing the stunting (height for age) among children < 2 years. The percentage of severe and moderate stunting in this HHS was higher than the finding of baseline survey (22.6% and 41.6% vs 17.1% and 38.2%, respectively (Figure 14)). However the percentage of stunting in this survey is lower than the national figure (44.2% was in LSIS and 41.6% is in our HHS). Although stunting in this survey is lower than in the national figure, the attention to improve stunting among children mainly children aged between months is needed because they are the most affected from stunting (Figure 15). Stunting in this age group could be a result of poor nutrient intake, poor knowledge on child feeding practices, poor care from the time of birth, chronic diseases like diarrhoea and pneumonia, burden of parasitic infection and poor personal hygiene and sanitation. The KII at community level viewed that the program had limited impact on nutrition due to the program focused more on monitoring on nutrition but not so much attention on treatment and demonstration. Therefore the promotion children 6-23 months access to variety of food groups and minimum acceptable diet through the promotion of home gardening, improving the mother s knowledge on young child feeding practices, promotion of good personal hygiene and sanitation and regular distribution of deworming to children under five years are essential to continue in Bokeo. Protection and management natural food resource such as nurturing and protection NTFPs, domestication of wild varieties of NTFPs, or fish conservation could be other alternatives to increase variety of and access to food. Figure 15: Stunting in three target districts Figure 16: Stunting by age in month and target district 58

78 Effectiveness of program s approach employed The designed approach employed for the program activities in order to improve the access to health service in communities: outreach activities and health events and building SBA to work in communities fits with the situation of target districts and provided a significant impacts on the access to health services in community. As we know target districts locate in mountainous areas and have poor public transportation so the access to health services is difficult. In addition the practices of ethnic community members deter the access to health services at health facility, for instance ethnic women dislike to consult with male health providers and ANC visit, deliver and PNC at health facility due to they are shame to allow others to see the parts of their bodies, also husband played an important role in decision making for household members to seeking health care at health facility: husband do not like to allow women to give birth at health facility and bring children to vaccinate. So that the outreach activities, health events and SBA working in community as volunteer held in villages are appropriate approach to employ (FGD and KII respondents). The local government partners totally supported the health day events organized in villages and they stated that if government had some financial support they would replicate this in communities by themselves (KII respondents). In this program government counterpart had joined meeting and provided some supports and advices and solved any problems when the program faced with difficulties. Communities also supported the health events in village and acknowledged the plan activities (KII respondents and FGD). They wished Plan to continue program activities in communities and encourage communities to use health services. However there were some constraints concerning the implementation of program observed such as the implementation of activities in communities was sometime delay due to the late communication between government counterpart and Plan staff based in province. Sometime outreach activities and health events did not reach to the resettlement and scatter villages. At commune level, there was a VHC that composed of 4-5 people and village facilitators to assist the program activities and collect some vital information in communities however the health staffs admitted that the VHC did not perform their task as planned only head of village and VHV work. 4.3 Monitoring and Evaluation The Program documents stated that annual and quarterly meetings with the PHD and key government counterparts will ensure opportunities for reflections, planning, and adjusting the project plan as may be necessary. Every year, during these meetings, target villages and budget planning will be reviewed for following year s initiatives. The monitoring framework employed by the project will conform to government policies and adapt with future modifications of the policy regarding sanitation in future. For the foreseeable future, the Program monitoring framework will comply to Model Healthy Village, Development Village, and School of Quality standards with the addition of ODF as a condition. Specific project indicators, especially for measuring project initiatives to promote greater leadership on the part of women and socially excluded groups. The WASH Program had established a monitoring system to keep track on all indicators. The star topics indicators monitoring system were particularly developed to a better stage compared to other indicators. The Project uses both quantitative and qualitative information 59

79 to monitor progress. However, the qualitative data was difficult to collect and had not been recorded and reported properly. The WASH reports mixed up data on indicators under objective 1 and objective 2 regarding the conditions of ODF and a number of officially declared ODF. As it had been found in the evaluation of the pilot project evaluation, there is complexity in using the system as the indicators (star topics) have both physical and behaviour indicators of achievement measured by both qualitative and quantitative means of verification. We believe that there are many indicators that would take a lot of time to collect data in order to assess its progress. For example, indicators numbers: 1.3) Number and % of episodes of diarrhoea reported in sample target villages; 1.4) % increase among women and children who know least three critical times to wash their hands (target 80%), 1.5) % of reduced costs for medical expenses related to poor sanitation (MNCH-N baseline WASH), 1.6) % increase among women and children who know at least two reasons that cause diarrhoea (target 80%); 1.7) % reduction of number of sick days from work or school resulting from being sick (with diarrhoea). To collect this information would take time and demanding for both adult, children and data collectors. Monitoring and review processes of the Project do monitor participation of women and men within Project activities. This is reflected in monitoring tools used which segregate gender information and monitors the roles of women and children in WASH. At present, Project monitoring does not yet monitor PWD. 60

80 5 Impacts and synergy Impacts at the household level In almost all FGDs, both women and men shared the view that Plan supported CWSs (including those supported by other agencies) did reduce women s burden. Women are responsible for fetching water, CWS reduced time and energy women have to spend on collecting water. Furthermore, whether this is intentional or un-intentional, villagers were able to connect water to their dwelling (PVC pipe or plastic pipe) which freed up women from water collection duty. However, this practice is not formally recognized and officially accepted. The practice can have negative impacts on the poor who could not effort the pipe or HH far away from the public water points or those located at a higher location within the village. The pipe distribution water system is being experimented in five communities in Attapeu province. The clear benefits include reducing women burden in collection water and convenient for supplying water to toilets. FGDs results show that convenience of water supply to the toilet is reported to be one of the key factors for the toilet use and cleanliness. FGDs respondents also reported that lack of water means toilets are left dirty and smelly-the unpleasant environment that no one want to come near it. In terms of gender equality, it is true that CWSs does alleviating women s workload, but it does not change the power relations between men and women per se. It can be argued that as a result of the reduced workload women are free to engage in other activities that are empowering them. These activities can include joining in income generating groups, engage in learning activities. Figure 16 below shows percentage of persons responsible for collecting water for HH use. It is clear that women are the bearer of this duty. We don t have baseline data to compare change in the sharing of this duty but it shows women are still taking on the HH chores as the norm would suggest them to be. Figure 17: Persons responsible for fetching water The MNCH-N program has brought about positive changes in knowledge and behavior concerning mother and child health among the beneficiaries though with limitation in term 61

81 of behavioural changes. The FGD participants acknowledged that they learned about the benefit of vaccination form health day events and outreach activities. Although secondary data and HHS indicate that the access to health services among women and children had increased over time during the project life, FGD participants revealed that many villagers did not take their children to get vaccination; they believed the children will get sick after being vaccinated. Some households intentionally left home when the health events took place in villages. Women preferred to bring children for vaccination but husband disagreed for that performance. This scenario implies that more health education concerning the adverse event of vaccination related to health should be emphasized to warrant all target children get vaccinated and head of household is the target for this health education. Regarding the ANC and PNC services, FGD participants reported that many PWs sought ANC visit at HCs, some PWs wanted to go for ANC visit at HCs but they could not go alone due to the distance and bad road conditions. They needed their husband who speaks better Lao to accompany them but their husband was not always cooperative as they preferred to go to the field over accompanying wife. Most PWs still delivered at home particularly ethnic women because they were shame to allow others to know about their reproductive health conditions and shy to let other see their secret part of the body. Sometimes they asked health center staffs to come assist at home. Some husband did not allow their wife to get deliver at health facility. PW would delivery at health facility just only when they felt that the delivery faced difficulties (FGD). Even with free delivery many ethnic women still did not use services at health facilities. Plan s support to build SBAs from local high school graduates and work in the community responded the cultural and social needs of the community. The FGD results are consistent with the results of HHS as the high percentage of mothers had ANC visit 4 times (47.2%) and most of them delivered at home (66.7%) and it was assisted by relative and friend (62.2%). Moreover pregnant women still worked hard in the field as normal people because elderly people perceived that the more working hard the more easily delivery will be done. In addition PW did not have a special food for themselves they will eat as normal people and good food would be shared with all family members. SBA and health center staffs should continue to provide health education to PW and husband concerning to the danger signs of pregnancy and delivery. Results of HHs indicates that the knowledge on danger signs of PW and PNC was still low (knowing 2 critical signs of PNC 39.6%, knowing 3 critical signs of PNC 18.0% and knowing critical sign of labor 18.0%). Many KII respondents said that awareness of general health care amongst villagers remains low. Health care and health treatment seeking behaviors had changed for the better. However, a large number of people still did not see the importance of health care services (cited by FGD participants). The KII respondents noted that mothers did not take a good care of their child health. They said that even when baby had a fever, cry and did not eat properly mothers still thought that was normal for babies and did not need special attention. It is important to note that during the cultivation period villagers left home for work in the field and missed out on health events in the village. For the next intervention this issue should be taken into account. Informed communities in advance needed to be ensured (KII field respondent). Generally speaking, the MNCH-N program has shown the positive impact on the gender equality and child right at the household level. Now women can represent the HH in the 62

82 village meetings. In many villages, women become members of committees while children participated in the activities in school. It was reported that at home women and men have equal right. Impacts at the community level The projects have generally brought about positive impacts on the lives of target communities. Hygiene promotion activities in schools and communities were reported to have resulted in positive changes at the community and school levels. These changes included cleaner surrounding environment around people house, school premise as well as common places like village meeting hall. Almost all FGD participants reported that positive behavioral changes concerning using toilet, washing hands with soap, drinking safe water and keeping environment clean were evident in their village. Almost all of the participants were aware of the importance of using toilets, washing hands with soap at critical times, drinking safe water and keeping surrounding environment clean. However, the results were varied in terms of converting this knowledge to behavior change (as outlined in the previous section). The results of these positive behavioral changes were also reported by FGD participants. They said in general health conditions among community members have been improved such as a reduction in a number of episodes of diarrhea in their communities. As we know that diarrhea is one of the main causes of malnutrition. So that in this HHs the achievement of MNCH-N program in addressing the wasting among children < 2 years was observed such as the severe and moderate wasting had decreased from 6.5% and 13.7% in baseline to 4.3% and 9.5% in this end-line survey, respectively. Thus to ensuring the achievement of MHV and addressing the malnutrition: wasting the integration of WASH and MNCH-N should be prioritized. This integration of WASH and MNCH-N program can be scale up to other provinces of Lao PDR. The FGD participants claim is consistent with the PHD authorities report which indicated that incidence of diarrhea has reduced in the target communities and believed to have occurred as a result of Plan intervention. They were not sick as often as before (i.e five years ago when there was no intervention) due to improved hygiene practices mentioned above. As a result they said, they could save a lot of money from spending on health treatment. They were not absent from work as often as before so they were able to work and fetch for food in the forest. They were able to better provide food for the family. These changes were tangible in the sense that villagers were able to realize themselves and thus were likely to improve over time. The intervention of the WASH project can be considered as a catalyst which triggers awareness and actions. Village health committees (including CLTS committees) will continue to function even not in the current form. Thus the positive changes will be maintained. Additionally, as the communities make progress economically they would afford TVs, travels to city, visits to relative in the cities. These will continue their exposure to improved health conditions which will also contribute to their desire to change. At the community level, the CLTS committees and village authorities interviewed reported that village regulations were in place and were used to re-enforce their work in promotion of hygiene practices. They also reported that sometimes they needed back up by district authorities in re-enforcing the regulations as the district authorities had more power and from experiences villagers seemed to listen to. Water safety plans (WSPs) and O&M guidelines were also in place in villages where Plan support water supply, even though some villages were not able to show us a copy. Protection 63

83 of water source catchment areas was well implemented and villagers were well aware of the importance of such protection. Only one out of six villages visited reported they had problem applying WSP (Houaynorkhom, Paktha District). Generally speaking, it was reported that the village authority there was very weak and was not cooperative. This was evident when the team visited the village. Villagers including the head of the village were walking around the village but refused to come to meet with us. Construction or rehabilitation of school WASH facilities and CWS were very relevant and remained relevant to the community needs. The intervention had brought about significant impacts on people s life at the community level particularly children and women. As will be discussed in greater details in the Gender and Social Inclusion section, women and children are the group responsible for fetching water. CWS bring water closer to their house thus reduced the workload of them significantly. FGD participants (women groups) reported that they were happy that they did not have to carry two budgets of water sometimes as far as 10 km for household use and drinking. Now they could save time and energy from this reduced workload. They were freer now to watch TVs, learning some cooking lessons from there, learning how to take care of kids from there, etc. in some villages where there are PPG program going on, women reported that they took part in the group activities learning and discussing about child care, health care, hygiene and related topics which improved their general knowledge about good life styles. The MNCH-N Program has also brought positive impacts at the community level and these impacts were well acknowledged by village authorities. The KII respondents stated that before children were frequently sick of diarrhea and some children died but now no children died of such disease. Authorities at villages level acknowledged that Plan activities helped materialized their mandates in encouraging health care seek at health facilities by villagers. They reported that as a famer themselves it had been difficult to encourage other to change behaviors; outreached activities delivered by DHO and PHD staff under Plan s support were very powerful in encouraging positive behavioural changes. The village authorities proposed a continuation of supported activities in their villages. In addition, children were very active in participating in the parenting education session organized in village. After the program intervention, the coverage rates of vaccination, contraceptive use, ANV and PNC visits within 7 days and 6 weeks had increased, particularly among WRA and children under-five years of age. The program had created substantial impacts on primary health care services among target beneficiaries through outreach activities, mobile clinics integrated with EPI, quarterly health days and children days. The majority of beneficiaries in target districts accessed to vaccination, deworming, vitamin A, growth monitoring, iron supplementation, family planning, ANC, and PNC. More women had better knowledge on critical signs of child illness that need to seek care immediately and the danger signs of PNC and labor course of delivery that require care at health facilities after participated in the parenting orientation activities led by trained health staffs and village facilitators. Plan had contributed to improved quality of MCH services in communities. Plan has supported 25 ethnic women with high school graduation to study SBA. Now they completed and voluntarily worked as SBA in communities and HCs. These women had obtained knowledge and competency to provide health care services in their own community. However, not all had been recruited as government staff yet. This intervention was well regarded by FGD participants as useful and user-friendly. The SBAs were easy to talk to, no language barrier; 64

84 and cheap sometime free. Therefore, this addressed the issues of shyness and long distance travel to health facilities. Moreover, training on Parenting Orientation Modules concerning MNCH-H was implemented in community by trained health and none-health staffs (education, Social and Welfare, Cultural and Information, LWU and Youth Union of provincial and district levels). This activity has made VHVs, VHCs, and village facilitators confident in providing services concerning the MNCH-N in their community. Impacts on the institutional level The program has improved the quality of health care services. Through the several short training courses on MNCH-N activities and long training course on SBAs the PHD, DHO and HC staffs and trained SBA had become more confident and competent to provide quality health services. Health facilities including 11 health centers and 02 district hospitals were supported with WASH facilities in order to provide healthy environment and cleanliness of the facilities. In addition, Plan provided enabling environment for delivery of MNCH-N services at HCs by expansion of two HCs such as the room for MCH services, labour room, and waiting room (cited by Plan staff) and equipped with necessary medical equipment for delivery. Through the capacity building training and study visit, health staffs at all levels gained more experiences and now feel more confident in performing their tasks. PHD and DHO staffs expressed that the computer skill is new for them but now they can work properly for the daily work and other purposes. PHD and DHO staffs also have capacity to supervise the program activities in communities such as data collection, parenting orientation on MNCH-N, outreach activities and other health events. Beside that health center staffs discussed that they now can develop a micro-planning for the program activities without the assistance of DHO so that they said that they even have confidence in leading the health event in community by themselves and if it is possible they wish to do so. The evaluators think that the application of PCA in communities from the start of project for identifying health problem has brought an important positive impact on the solidarity and unanimity between health staffs and communities towards the mutual addressing of health problems in communities although Plan had played an important role in designing the appropriate program activities in corresponding with the identified problems. Impacts at the local government levels The project has made the most important contribution at the district level. This included knowledge and competency strengthening for district officials. District officials interviewed named trainings they participated in as CLTS TOT training, basic gender principles training, gender WASH monitoring toolkit training and disability right, inclusion and equity. The officials shared the view that the trainings not only exposed them to the content i.e. gender equality principles or gender wash monitoring indicators, but also the facilitation and promotion skills. Some even proposed the ideas of more support in the form of on the job coaching for example when conducting CLTS sessions. Most importantly, the officials said they were more competent speaking at the public settings such as community meetings or at the workshops or formal meetings. They said in the formal meetings before even if they had some ideas but were too shy to share them. Now they have been more active and participatory in such settings. 65

85 In terms of organizational structure such as DWTs, experiences on multi-sectoral coordination and planning had had been improved or initiated over the project period. This coordination and cooperation particularly between education and health sectors has contributed to effectiveness of CLTS such as improved hygiene practices at the community level or increase in a number of ODF villages. Since improved health conditions among target communities depend on this multi-sector cooperation, DWTs have made important improvement both in terms of teamwork and achievement of the team mandate. The introduced organization such as DWTs were unlikely to be sustainable and the one of the key reasons is the lack of financial capacity from the government system to continue to make it functioned. Interviews with PHD and DHO all pointed to the same conclusion. They admitted that the government at least at the provincial level would not have financial capacity to fund DTWs and continue Plan initiated activities related to CLTS. Therefore it is unlikely that DWTs would continue to exist and function as it has been until now. Through a quarterly meeting the PHO, DHO and HC staffs had a good culture for exchanging experience amongst them and mutual reviewing all important issues. Also through the monitoring and supervision of the program heath staffs had better communication and coordination with local committees at village level including SBA working as volunteers, VHC and village facilitators. These performances promoted the implementation of program activities at village level going smoothly and ensured the implementation going to the right way and sustainability. Synergy Synergy between the WASH and MNCH-N programs occurred when both programs presented. The two programs were complemented and closely linked. The WASH program provides access to WASH facilities in schools and communities at the same time promotes proper hygiene behaviours. The MNCH-N program, on the other hand, provides access the mother and child health care services. The two are complementarily contributed to disease prevention and treatment. Integration of MNCH-N and WASH programs is well acknowledged as an important approach to contribute to MCH and nutrition outcome and impacts. At the national level the integration has already been performed and set it as convergence plan. It involves 4 sectors: education, health, WASH and agriculture. The community members reported that before children were frequently sick of diarrhea and some children died. The villager authorities reported that over the last three years there was no case of under-five mortality. The integration of WASH and MNCH-N programs has given a positive impact not only on the acceleration of the development of MHV but also addressing malnutrition among children < 2 years particularly wasting. From the field observation, the team found that very few sanitation facilities were constructed in villages where there was no WASH program present. At the same time, more women and men reported seasonal diarrhoea outbreaks in villages where there was only MNCH-N program presented. The above impacts are a synergy effect of integration between WASH and MNCH-N programs. As indicated in this HHS the hygiene and sanitation in villages had been improved: 78% of target villages achieved ODF, 34.5% of mothers had better knowledge on how to dispose of child feces properly, at least people knew about three critical times to wash their hand (Meung 23%, Paktha 12% and Pha Oudom 35%) and FGD participants knew the cause of diarrhoea correctly. The National Nutrition Strategy 2009 emphasises the causes of under- 66

86 nutrition including adequate amount of diverse and quality food intake and proper childcare and feeding practice. However, there was a limited impact on nutrition element even though integrated into the MNCH-N program. The integration of WASH and MNCH-N programs had created positive impacts not only on the acceleration of the development of MHV 26 but also addressing malnutrition among children < 2 years particularly wasting. Criteria for the MHV status included both WASH and MNCH-N program. 27 Implementation of two program activities in the same village also promoted the development of MHV faster. In Bokeo province the two programs activities had been implemented in target districts so the evidence showed that the program greatly achieved the MHV (67.6%-23 out of 34 target villages) and 11 villages are in process. 26 1) implementing comprehensive health education availability of information board or loud speaker, 2) provision of clean water and environment in collaboration with concerned unit, 3) implementing a process of three hygiene, 4) having trained VHV on health promotion, prevention and able to solve basic health problem, 5) contribution to promote the vaccination among target children and women and reaching to 90%, 6) implementing a process that supports safe pregnancy, delivery ANC visit 4times, delivery and PNC at health facility and attended by SBA, 7) implementation of integrated package of mother neonatal and child health services, 8) having functioning village drug kit in case the village is not under catchment areas of health center, 9) transferring patient to health facility in time, and 10) having village health committee to promote health and prevent diseases, and recording and reporting all birth and death occurred in a village. 27 H-SDP - ADB III, Ministry of Health, Manual on establishing the model health village. 67

87 6 Sustainability 6.1 Sustainability of WASH In this evaluation, Sustainability is the ability of prevailing local structures, processes and people to continue their role and functions after the withdrawal of all forms of support from Plan. As a result, the WASH program activities, outputs, outcomes and impacts would persist during an extensive time period. The evaluation documented the presence of the conditions and factors associated with WASH sustainability and the likelihood or prospects for sustainability. School WASH Infrastructure As a result of the school break, we could only manage to make site observation at three schools. We observed functionality and cleanliness of the WASH facilities. We found low quality construction work. (i.e. leaking water storage). Sustainability in its simplest sense is the durability of the structure. Plan staff admitted that supervision role for construction work in schools had been transferred to the government counterpart. They now acknowledged that the transition means an increased risk of poor quality work. Plan and the counterparts had addressed the issue internally. The maintenance of the completed facilities is the responsibility of the village education development committees (VEDCs) and the school principles. The project established the committees to oversee the functioning and maintenance of the infrastructure. The committees reported that the school administration fund was the only source of money to be spent on maintenance of school wash facilities. The committee reported that minor maintenance could be done by the CLTS committee members, the technical persons. Technical capacity for maintenance of these facilities has been developed by provision of O&M training. Unless the technical person moved out of the village immediately and someone else does not take up the role, the maintenance mechanism will remain there. Hygiene in schools Sustainability of hygiene promotion in schools rested on the integration of hygiene promotion activities into school curriculum and the hygiene promotion activities led by teachers and model/volunteer students. These activities were based on the UNICEF hygiene promotion toolkit called the Bluebox. FGDs with children revealed that they practiced hand washing with soap once a day, three times per week when the school semester was on. There is no standard in terms of how many time per week is good, but students seemed to know well for good hygiene practice such as 7 steps of hand-washing. According to the education officials interviewed hygiene practice will continue to be implemented even without financial support from outside. Therefore, this is likely to be sustainable. Funding for hygiene activities such as provision of soap is the key to its sustainability. According to education officials interviewed, the schools that meet the government SoQ criteria would receive encouragement fund (1000kip per student per year). However, none of the schools that had met the criteria had received the money yet. The possibility that the money would be made available is very limited as confirmed by the MoES. Other schools would fund the hygiene practice activities by the school administration cost either allocated from district education office or collected from student parents. 68

88 Institutional sustainability The Institutional structure developed under the project related to school WASH is the establishment of DWTs, VEDCs, CLTS committees and involvement of village chiefs and village cluster heads. Over the project period, these entities were actively involved in the school wash activities. As discussed earlier, DWTs may not be sustainable in its current form therefore; maintenance of the school facilities may need to be re-arranged after project had end. Sanitation infrastructure Construction of the latrine by self-investment was well monitored and reported. According to KII participants, pour flush type of toilets were the most popular to villagers. From the project s experiences, dry pit toilets were prone to extreme weather events. This becomes the main reason for some villages to lose their ODF status. Therefore, mmaintaining and keeping these facilities is critical to sustaining changed behaviours among children in schools and villagers in the communities. The key to sustainability of CLTS are continuous follow-up among households to ensure maintenance of toilets and sustained use. However, for some villagers dry pit toilets are the first stage toward permanent toilets. A critical factor therefore is the maintenance of the dry pit toilets which are structurally unsustainable especially in challenging geographical and weather conditions. A distinction has to be made between the maintenance of just the toilet-seat and building, and the system to evacuate and treat excreta safely. Behavioral changes FGD participants reported that the deeply rooted habits of OD started to change now partly due to understanding of the negative health impacts of the OD and partly due to the convenience of having a toilet at home. Children groups reported that they were taught at school to use the toilet and wash hands with soap after using the toilet. Some were able to apply the practice at home, some even persuaded their parents to build and use toilets and practice hand washing with soap. The majority of men and women of FGD participants reported that they used toilet and wash hands with soap all the time. However, there were some who admitted washing hands without soap or not washing hand at all. Institutional Plan is striving to ensure local ownership and community-led processes in relation to WASH. Local governance structures such as VEDCs and CLTS committees were established to empower community participation. However, across the three districts visited, officials both Plan and government counterparts, reported that community ownership remained an issue concern. At community level, even village authorities, still believed that WASH facilities belong to Plan. Villagers were under the impression the project is owned by the district or provincial departments. Support from outside for CLTS committees is unlikely to be sustainable. At the moment, there still strong commitment in following up from DWTs. They had achieved a satisfactory result as discussed earlier. However, the current support will cease when the project had ended. Sustainability of the CLTS committees is unlikely unless the governance structure is supportive of it. This can include integration of the initiative into the district WASH plans and budget. However, interviews with PHD officials and DWT members reveal that there is no particular budget line for the CLTS initiative. Adoption of the CLTS approach by local government 69

89 institutions can lead to sustainability of the initiative and demand for sanitation at the community level. The project s exit strategies The project s strategy to phasing out has integrated in the design by setting a WASH team at the district level. At the national level, Plan had done quite extensive work. Figure 2 below shows some of Plan s engagement with other development partners in attempts to promote adoption and expansion the CLTS approach. For example, the applied approach and developed training manuals are shared at national level with the NamSaat, the WSP, SNV, and with Plan s country Program. As well, representatives from PHD Bokeo were supported to experience first-hand the project experiences of SNV in Savannakhet where such experience and knowledge is expected to be shared at all Health ministry levels. Water supply system management Water use systems vary from village to village. One of the alarm signs in terms of sustainable use of water is water distribution. CWS distributed water by a water point usual shared by 5 to 10 HHs. Accordingly, many village authorities interviewed allocated a water point and the HH using it as a water group. If minor maintenance was needed i.e. water tap replacement, the group members would have to share the cost. This mechanism was widely adopted and seemed to work well. In other villages, a pool fund for the whole community was established and water fee was collected in advance. However, village authorities interviewed admitted that were not able collect the fee from all the HHs. This was well confirmed by the HHS conducted for this evaluation. Figure 1 below shows percentages of people who pay the fee by three categories: regular, sometimes and never between target communities with Plan s support CWS and none-plan support system. The proportion of regular fee payers is just 37 % which is quite low (see Figure 1). Figure 18: Fee payment by Plan and none-plan supported CWSs This result is vastly different from the rate at which it was report in the 2015 annual report (80%). When compare the Plan s CWS with other villages supported by other organisations the result is not significantly different. There were 6 out of 8 target villages where Plan supported CWS sampled for the survey. The results from the HHS show a limited difference in terms of fee payment between Plan s CWSs and the rest of the target villages where water systems also exist. From interviews with the project staff, we learned that non-plan supported 70

90 water supply systems had been established before Plan started the intervention in the area. Some sort of water committees also exist. All 8 villages have started implementing village regulations that include catchment protection in line with Operations & Management (O&M) and Water Safety Plan (WSP) guidelines. The training was conducted on January This means that the committee members have just been trained and likely that they would not enough time to put all the knowledge recently gained into effective application. Institutional sustainability A large part of sustainability of water facilities depend on community ownership and participation. At the community level, Plan International Laos and DWTs form and train CLTS Committees two weeks after triggering. Committees replace natural leaders and comprise five to seven community leaders who collect data, promote behavior change, and report progress to DWTs. In the interviews, the committee members reported they did not have a written plan or reports. In some villages, the team observed that when asked how many toilets were built this year, the member counted fingers to come up with the number of toilet built. This implies that they did not have any written form of paper (it is a norm that they bring notebook or any formal records to meetings such as meeting with the team). When asked if incentive is important for water committees to function well and be sustainable, the members indirectly replied yes it is important. At the moment, water committee as well as all other committees established at the community level rested on the premise of volunteerism. In fact, this is a great myth. Some would volunteer to the collective good of their communities. But it is unnatural to fund enough volunteers in every community. Some water committee members admitted that they were selected by the community and had to fulfil the obligation rather than by a genuine volunteer per se. However there are some constraints might be contributed to Program sustainability. Ownership of government counterpart, high rotation of staff will need more training, workload for HC staffs and community leaders, lack of budget for organizing health events in villages so it will lead to few health events held in villages or may be none and villages are scattered. These constraints can be threat to the sustainability of the MNCH-N Program. As a result of the school break, the team could only manage to make site observation at three schools. The team observed functionality and cleanliness of the WASH facilities. The team found constructed facilities to be low quality (i.e. leaking water storage). Sustainability in its simplest sense is the durability of the structure. Plan staff admitted that supervision role transferred to the government counterpart mean increased risk of poor quality construction work. While DWTs and CLTS committees promote good inter-departmental cooperation at the district level and good team work at the village level, the sustainability of these structures depend on the formalization of the structures and the institutional change within the local government structure. For example, there must be budget arrangement and commitment by the government to carry on with the initiatives. The problem at the moment is that there is no downward accountability from the government side. In general, the government system does not respond to community needs. The system is either financial or technical incapability to do so. 71

91 At village level, the situation might be different as the CLTS committees have the formalized and recognized (by district authorities) village regulations. As long as the communities recognize and accept the authority/ validity of the regulation, the CLTS committees even if not formalized can continue with encouragements to households as regards maintenance and use of toilets, keeping the environment clean, and other guidelines contained in their village regulations. At this level that Plan should refocus their effort on. It is unlikely that DWTs will continue to function after the financial support from Plan has ceased. Therefore, CLTS activities must remain and be carried out by VHC members (in case CTLS at community level is integrated into VHC). It is likely that working on fewer committees will ensure sustainability. Review of the international experiences, it is widely acknowledge that that sustainability of school WASH facilities remains a great challenge. The limited budgetary resources allocated to school operation and maintenance, particularly in rural schools, means that few schools have any funds available to pay a caretaker or cleaner for regular cleaning and maintenance of school WASH facilities the standard management model for school WASH facilities in most developed countries. This experience was found to be the case with the UNICEF WASH in School Program 28. For Plan s intervention, school WASH leans its sustainability on VEDCs. From interviews with VEDC members the team found some inconsistency is the management and maintenance system. In some villages, a separate fund was kept for maintenance of the facilities. While other villages rely on the school administration budget allocated to school by district education office. Other source of the maintenance budget is said to come from the prize money given to school that meet SoQ standards. This has not been confirmed by all village authorities interviewed to be the case for all schools. Sustainability of CLTS committees at village level remains a great challenge. One of the reasons is that at the moment they are not functioning as they supposed to be. From interviews with CLTS committee members, the team found that they have limited skills in encouraging other villagers to improve sanitation. In many villages, the committee members themselves had not built toilet yet. Another reason is the lack of incentive/reward for the effort for collective gain. This together with the fact that they are not fully volunteered in becoming the committee members can lead to unsustainability of the initiative itself. Support from outside for CLTS committees is unlikely to be sustainable. At the moment, there still strong commitment in following up from DWTs. They had achieved a satisfactory result as discussed earlier. Sustainability of the initiative is unlikely unless the governance structure is supportive of it. This can include integration of the initiative into the district WASH plans and budget. However, interviews with DH officials and DWT members reveal that there is no particular budget line for the CLTS initiative. Adoption of the CLTS approach by local government institutions can lead to sustainability of the initiative and demand for sanitation at the community level. A critical factor that must be considered in sustainability of CLTS is the nature and type of toilets used. Experience from other countries, (notably in Cambodia where conditions are similar (lack of water, high rainfall during the wet season, high poverty rates)) show that drypit toilets are not sustainable from a durability perspective especially in the wet season. In project target areas, some villages are located in hills and toilets are built on slopes at the 28 UNICEF WASH programme evaluation

92 fringes of the village. Where toilets are dry-pits, there is a constant and high risk of toilet structure collapse and pit-overflow from rainfall. In flat-land villages, the risk is pit flooding and over-flow. Both consequences present an unpleasant dilemma for HHs and may result to OD during the wet season repairing their dry-pit toilets in the dry season. Improved latrine designs addressing the existing risks in target villages would address these issues in the short term. However, for the long-term and for villages that are not at risk of relocation, encouraging conversion to durable pour-flushed toilets is the ideal response. This however, requires that HH are economically- capable to convert to such type of toilets and/ or that lowcost durable toilets are accessible to these HHs. A large part of sustainability of facilities and personal and household hygiene practices depend on community ownership and participation. At the community level, Plan International Laos and DWTs formed and trained CLTS Committees two weeks after triggering. Committees replaced natural leaders and comprise five to seven community leaders who collect data, promoted behavior change, and reported progress to DWTs. In the interviews, the committee members reported they did not have a written plan or reports. The team observed that when asked how many toilets built this year, the member counted fingers to come up with the number of toilet built. This implies that they did not have any written form of paper (it is a norm that they bring notebook or any formal records to meetings such as meeting with the team). Figure 19: Fee payment by Plan and none-plan supported CWSs Community member s participation is also key to sustainability. In this respect, the team look into active contribution of villagers to maintenance of the facilities-one way is by paying fee to the water fund. The results from HHS show a low rate of fee payment contributing to water fund at the community level, 37% pay the fee regularly. This result is vastly different from the rate at which it was report in the 2015 annual report (80%). When compare the Plan s CWS with other villages supported by other organisations the result is not significantly different. There were 6 out of 8 target villages where Plan supported CWS sampled for the survey. The results from the HHS show a limited difference in terms of fee payment between Plan s CWSs and the rest of the target villages where water systems also exist. From interviews with the project staff, non-plan supported water supply systems had been established before Plan started the intervention in the area. Some sort of water committees also existed. 73