End-Term Evaluation Of AMUA Project

Transcription

1 End-Term Evaluation Of AMUA Project Final Report April 2013 Participatory Development Consultancy (PDC) P.O Box , Nairobi, Kenya AMUA Project End Term Evaluation Report (April 2013) Page 1 of 56

2 Table of Contents List of Tables and Figures... 4 Acknowledgements... 5 List of Acronyms... 6 Executive Summary Introduction Access to Sexual and Reproductive Health Services Marie Stopes Kenya Country Response The AMUA Project Objectives of the Evaluation Methodology Study Design and Methods Study population Sampling Method and Sample Size Determination Household interview Household Interview Key Informant Interviews Focused Group Discussions Observation Most Significant Change (MSC) Technique Methods of Data Management Enumerators Selection and Training Involvement of Marie Stopes Kenya Pre-testing of Data Collection Tools Mobilization of the Communities for the Evaluation Exercise Ethical Considerations Data cleaning Data analysis Findings and Discussions Demographic Characteristics of the Respondents Access and Utilization of RH and HIV&AIDS Services Utilization of AMUA Services Family Planning Commodity Security Adherence to Government Policy Guidelines and Protocols AMUA Project End Term Evaluation Report (April 2013) Page 2 of 56

3 3.3 Behaviour Change and Demand for RH and HIV&AIDS Services Enhanced Capacity of Providers to Offer Services Coordination between Public and Private Sector Monitoring and Information Sharing Relevance, Effectiveness, Efficiency and Sustainability Relevance of AMUA Project Effectiveness of AMUA Project Efficiency in Interventions Sustainability Challenges and Lessons Learnt Challenges Lessons Conclusions and Recommendations Conclusions Recommendations Annex 1: People Consulted AMUA Project End Term Evaluation Report (April 2013) Page 3 of 56

4 List of Tables and Figures List of Tables Page Table 1: Sample size allocations.. 19 Table 2: Sampled health facilities. 20 Table 3: Socio-demographic characteristics of the clients 25 Table 4: Source of current FP methods 27 Table 5: Services Obtained During Last Visit.. 31 Table 6: Service received at last visit to AMUA facility and ability to pay for the service 31 Table 7: Comparison between utilization of Condoms and Injectables in different sites 32 Table 8: CYPs realized during project life 46 List of Figures Figure 1: Distribution of KfW supported AMUA health facilities.. 20 Figure 2: Distribution of respondents by region. 24 Figure 3: Percentage of AMUA clients currently using various FP methods 26 Figure 4: Cost rating by clients for FP/RH services in AMUA clinics. 27 Figure 5: HIV services offered within AMUA network health facilities 28 Figure 6: AMUA Clients Perception of Integration in Clinics 29 Figure 7: Clients Opinion about AMUA Services.. 29 Figure 8: Last time client visited AMUA facility.. 30 Figure 9: Transport cost to AMUA facility.. 30 Figure 10: Branded AMUA clinic. 36 Figure 11: The 7Ps Approach to Marketing Health Services. 38 AMUA Project End Term Evaluation Report (April 2013) Page 4 of 56

5 Acknowledgements The Marie Stopes Kenya AMUA project end term evaluation was conducted in a highly participatory manner by a team of consultants from Participatory Development Consultancy led by Salmon Owii, Albert Oduor, Dr. David Soti, and Nick Oyoo. Many people put a lot of effort to produce this report, and it is not possible to mention all institutions or individuals who made valuable contributions in one form or another. However, special thanks go to our team of supervisors namely Herman Jaoko, Dominic Obaye, Charles Angira, Duncan Ager and Moses Ooko. We are also grateful to the Marie Stopes Kenya team for their guidance before and during the evaluation. We are especially grateful to Mr. Vitalis Akora, the MSK Country Monitoring and Evaluation Manager and Dr. Moses Mwaniki the AMUA Project Manager for their guidance and support throughout the process of conducting the evaluation. Our appreciation also go to the AMUA regional coordinators and other MSK staff in Kisumu, Homabay, Kisii, Kitale and Nakuru for not only providing information but also facilitating mobilization of respondents and scheduling of interviews and discussions. Similarly, we are grateful to the team of 15 research assistants who worked tirelessly to gather data for the evaluation. We also acknowledge the District Health Management Teams from the focus regions for supporting the evaluation team in their work. The private providers and their staff, as well as community health workers also made invaluable contribution, and to them we are greatly indebted. We are also most indebted to all those who participated in the study namely; the key informants, the mothers, the community members, women and men who participated in focused group discussions, community leaders and representatives of community structures involved in the evaluation. We are grateful to all for their contribution and enthusiasm. Finally, we are grateful to KfW not just for the general technical support, but also for the financial support that enabled MSK to commission and facilitate execution of the AMUA end term evaluation. AMUA Project End Term Evaluation Report (April 2013) Page 5 of 56

6 List of Acronyms AIDS - Acquired Immune Deficiency Syndrome ANC - Ante Natal Care AOP - Annual Operational Plan ARV - Antiretroviral ASRH - Adolescent Sexual and Reproductive Health BTL - Bilateral Tubal Ligation CCS - Cervical Cancer Screening CHC - Community Health Communities CHW - Community Health Workers CME - Continuing Medical Education CSO - Civil Society Organization CTU - Contraceptive Technology Update CU - Community Units CYP - Couple Year Protection DHMT - District Health Management Team DRH - Division of Reproductive Health DRHC - District Reproductive Health Coordinator FBO - Faith Based Organizations FGD - Focused Group Discussion FP - Family Planning GoK - Government of Kenya HFC - Health Facility HIS - Health Information Management HIV - Human Immunodeficiency Virus HRH - Human Resources for Health HTC - HIV Testing and Counseling IEC - Information Education and Communication IUCD - Intra-uterine Contraceptive Device KEMSA - Kenya Medical Supplies Agency KDHS - Kenya Demographic and Health Survey KfW - Kreditanstalt fur Wiederaufbau KII - Key Informant Interview K-MET - Kisumu Medical Education Trust AMUA Project End Term Evaluation Report (April 2013) Page 6 of 56

7 K-REP - Kenya Rural Enterprise Programme KURHI - Kenya Urban Reproductive Health Initiative KWFT - Kenya Women Finance Trust LAPM - Long Acting and Permanent Methods MDG - Millennium Development Goals MFI - Micro Finance Institutions MOH - Ministry of Health MPAC - Medical Post Abortion Care MSC - Most Significant Change MSI - Marie Stopes International MSK - Marie Stopes Kenya NGO - Non-Governmental Organizations NHSSP - National Health Sector Strategic Plan NNAK - National Nurses Association of Kenya OBA - Output Based Aid PDC - Participatory Development Consultancy PMCG - Project Management Co-ordination Group PMTCT - Prevention of Mother to Child Transmission PPP - Public Private Partnership PSI - Population Services International QTA - Quality Technical Assessments RH - Reproductive Health SF - Social Franchising SONY - South Nyanza (Sugar Company) SOP - Standard Operating Procedures SRH - Sexual and Reproductive Health STI - Sexually Transmitted Infection ToR - Terms of Reference VCT - Voluntary Counseling and Testing VHC - Village Health Committees VIA - Visual Inspection using Acetic Acid VILLI - Visual Inspection using Lugos Iodine YFS - Youth Friendly Services AMUA Project End Term Evaluation Report (April 2013) Page 7 of 56

8 Executive Summary Background In Kenya, the unmet need for Family Planning (FP) is at 25% which creates a gap between the actual and desired family size. This means that couples are having more children than they had initially intended to due to lack of access to modern Family Planning (FP) methods. The demand for SRH services exceed the supply by the government, hence the need for the private sector to augment the deficit through Public Private Partnership (PPP) and other mechanisms such as social franchise, with a view to expanding the range of FP choices for women of reproductive age. Since 2004, Marie Stopes Kenya (MSK) has been implementing AMUA project, and continue to do so in partnership with the Ministry of Health through funding from different development partners, and as at the time of the evaluation the network was working with 280 private health facilities. The largest component has been supported by a German based health sector support agency - KFW, and at the time of evaluation had a network of 167 private health facilities. The purpose of the project has been to expand and extend the AMUA social franchise network to increase access to and utilization of high-quality Reproductive Health (RH) and HIV/AIDS services in selected underserved areas in Kenya. The anticipated project outputs included scaled up access and utilization of long-term FP services and counselling; improved awareness, positive behaviour change and increased demand for long-term FP services and counselling; enhanced capacity of MSK and private practitioners to provide longterm high-quality FP services, information and counselling; and increased coordination and collaboration between the public and private sector service providers. The outputs were to be realised through scale up of private service provider activities under the fractional social franchising model of healthcare financing. In December 2012 an end term evaluation was commissioned to assess the relevance of the KFW-funded AMUA project component, its effectiveness, efficiency, impact and sustainability in the three focus provinces of Rift Valley, Nyanza and Western provinces. The evaluation employed both quantitative and qualitative methodologies. It was conducted in two phases with the first phase entailing review of program documents and other relevant literature to among others inform the development of evaluation tools, and the second phase entailing quantitative and qualitative data collection in the field using semi-structured questionnaires and discussion and interview guides. Structured questionnaires were administered to 747 sampled households in the 5 project defined regions namely Nakuru; Kisumu; Homabay; Kitale and Kisii all in Rift Valley, Nyanza and Western provinces of Kenya. At the time of the evaluation, the AMUA social franchise network was working with 280 private health facilities in the whole country, with the component funded by KFW, and which was the focus of this evaluation working with 167 private healthcare providers. A total of 25 health facilities were sampled, 5 in each of the 5 project regions around which the 747 respondents were identified. In addition, 38 Key-Informant Interviews (KIIs) and 10 Focused Group Discussions (FGDs) were conducted to gather information relevant to the evaluation, with the latter drawing discussants from women of reproductive age years who had been getting AMUA services. The study population AMUA Project End Term Evaluation Report (April 2013) Page 8 of 56

9 included project beneficiaries such as women of reproductive age (15-49 years), their households, males, Ministry of Health staff, and other partners that had been involved in the program. The quantitative data was analyzed using Statistical Package for Social Scientists (SPSS) software while qualitative data were analyzed using content analysis method. Access and Utilization of RH and HIV&AIDS Services The end term evaluation determined that all the AMUA services are available in many of the network health facilities. Even though FGD participants indicated that some of the services are also found in other private clinics, chemists, and government hospitals, they observed that AMUA clinics are accessible, the services are good with short waiting times, and have friendly service providers who offer good counseling sessions. The evaluation determined that FP services offered in most AMUA health facilities included FP counseling and FP methods such as injectables, pills, Intra- Uterine Contraceptive Devise (IUCD), Jadelle, and condoms. As such, AMUA clients use varied FP methods, with injectables being the most widely used and male sterilization the least used method amongst the clients. In addition, Marie Stopes Kenya s AMUA project is well recognized as a source of FP in comparison to other available sources with 60% (n=448) of the clients depending on it for their current FP needs. The majority of the clients (n=639, 86%) felt that the current available FP/RH services met the needs of the community. Generally, as envisaged during the project design phase, the AMUA project has increased access to quality and affordable FP services in the five focus regions and there are concerted efforts to deliver integrated services to clients. Optimal realization of this objective and integration has been affected by commodity insecurity for both FP and other services, as well as inadequate staffing at health facility level. The clients are generally happy with services, except for cost, which is sometimes compared to rates offered by facilities in other social franchise networks such as Tunza run by Population Services International (PSI). Going forward, there is need to make significant provisions for strengthening commodity security for FP and HIV/AIDS related services, and enhanced support to government structures for staff supervision and coordination Behaviour Change and Demand for RH and HIV&AIDS Services The AMUA project has provided information to the public regarding Sexual Reproductive Health (SRH) services offered at the various outlets through printing and distribution of Information Education and Communication (IEC) posters and brochures developed and endorsed by the Division of Reproductive Health (DRH), and through radio spots, talk shows on FP/SRH and one-on-one information sharing by both AMUA staff and Community Health Workers (CHWs). In the course of implementing the project, the project undertook a review of its demand generation strategy. Based on this review, the project embarked on an enhanced generation strategy which involved the development of specific messages and targeting strategies to reach the AMUA clients, namely the poor who can only afford to pay low, subsidized fees. The AMUA project demand generation objectives focused on increasing use/adoption of all FP methods, integrating Youth Friendly Services (YFS), including RH and contraceptive services within the existing AMUA Project End Term Evaluation Report (April 2013) Page 9 of 56

10 services, and increasing uptake and utilization of other services. The strategies employed have included interpersonal communication (CHW and peer educators) and various promotional campaigns. The demand creation activities have been partly effective. Over 90% of AMUA clients express satisfaction with services offered in AMUA clinics and nearly all clients would be willing to recommend services to others 95%. The demand creation activities have complemented each other fairly well and have yielded results with not only greater awareness for services, but also increased demand for needy women not able to access FP services at the time of need. Clients prefer seeking FP/RH services at the private medical facilities due to a number of factors including privacy, confidentiality, convenience, cleanliness of the facilities, and advertisements of the AMUA branding. Those seeking FP/RH services mostly do so after sharing with one another and after being reassured that the type of a particular service is good. Almost all clients interviewed (99.5%, n=743) admitted having ever heard of the term FP with a large majority (83%, n=621) acknowledging that they are currently using FP. The main reason why families use FP methods is so as to be able to space their families (58%, n=434) followed by the desire of not wanting more children (18%, n=136). Generally, despite the marketing strategies put in place to promote AMUA there is need for more innovative strategies that can ensure greater reach to more clients. There is also need for the network to periodically review the cost of its services so that franchisees remain competitive in the face of other competition, such as PSI s Tunza social franchise network. Further, efforts should be made to demystify FP and promote uptake of long term methods for greater Couple of Years of Protection (CYPs). Since health care, like any other economic good is unique in its core characteristics related to demand and supply, marketing strategies for FP as a health care service must, therefore, be uniquely defined, by being cognizant of its peculiar parameters. Deploying new social media for uptake promotion and market intelligence is critical in creating demand for FP. Such approach may include short text messages on FP through hand held mobile phones to registered clients, and developing knowledge based portal a hub for targeted clients. There is also need to critically interrogate the financing approaches. The concept of SF requires multifaceted financing approaches that are geared towards realizing cost-containment, and maximizing utilization of services by clients across income quintiles. Enhanced Capacity of Providers to Offer Services Capacity building of private providers has been a process that has involved several steps including: identification and recruitment of private health service providers; capacity assessment of providers; training of providers; ongoing mentorship; equipment supply to provider clinics based on capacity gaps and project priorities; monitoring; facilitative supervision; joint reviews; facilitation of linkages with Kenya Medical Supplies Agency (KEMSA) for commodity supply; support with AMUA buffer stocks; branding; and continuous support for demand creation. Assessment of capacity is done prior to recruitment to determine among others satisfaction of criteria and needs that can be addressed. The clinics enter an agreement with Marie Stopes Kenya to ensure quality provision of and access to a targeted range of services. AMUA Project End Term Evaluation Report (April 2013) Page 10 of 56

11 AMUA membership signifies both benefits and obligations. AMUA offers marketing, promotional and mobilization support to foster the demand side from clients. In return, the franchised clinics provide quality AMUA franchise services within their service mix, avail themselves for training, submit monthly reports, and adhere to clinical protocols and other agreed terms stipulated in the AMUA franchisee agreement. By and large, the capacity of all the AMUA franchisees have been greatly enhanced during the period under review and they are now delivering higher quality RH and HIV/AIDS services, information and counseling. Going forward, there are opportunities that an extended or expanded phase of AMUA can exploit, especially those coming with the county governments in the new devolved government system. The county governments will see governance centre and, therefore, services moved nearer to people thus allowing for prompt technical assistance and enhanced commodity security. The expanded phase will need to invest more on facilitating oversight and coordination roles by the government, especially at the decentralized levels. Coordination between Public and Private Sector In order to strengthen implementation and enhance access to FP services for poor women in rural areas, MSK has collaborated and nurtured partnerships with different likeminded stakeholders. The stakeholders include the Ministry of Health (MOH), National Nurses Association of Kenya (NNAK), MSI, KfW, clients, and private practitioners, who have been engaged through a Public Private Partnership (PPP) model. At the national level, the MOH provided overall stewardship and strategic direction to the programme through the Project Management Co-ordination Group (PMCG) chaired by the Division of Reproductive Health (DRH) head. In order to realize increased coordination and collaboration between the public and private sector service providers, AMUA has been integrated in health sector plans. At the service delivery level, the project adopted a PPP referral network model, whereby CHWs affiliated to AMUA franchised health facilities sensitize communities and create demand for RH and HIV&AIDS services, including long-term and permanent FP methods (LTPMs), and raise client awareness for the AMUA clinics and brand. Generally, efforts aimed at realizing greater coordination between these project stakeholders has seen increased equitable access to health services, improved quality and the responsiveness of services in the sector, improved efficiency and effectiveness of service delivery, enhanced regulatory capacity of MOH and strengthened partnerships in improving health and delivering services. In an expanded phase, there should be a focus on facilitating other partners to adopt and scale up some of the best practices that have emerged from AMUA project. Monitoring and Information Sharing The project has a Monitoring and Evaluation (M&E) performance framework that regularly assesses programme progress against key performance indicators, and also a logical framework. Programme outputs are monitored on an ongoing basis through facilitative supervision, field visits, client exit interviews, Quality Technical Assurance (QTA), periodic team and management review meetings, periodic project reviews and regular reports. The AMUA project has also established a strong Quality Assurance AMUA Project End Term Evaluation Report (April 2013) Page 11 of 56

12 (QA) mechanism. The project provides routine supportive supervision to franchisees, with more visits to partners that present gaps in quality as determined through staff feedback and also through routine internal and external QTAs that employ use of standard MSI QTA protocols. Franchisees are also incorporated into MSI annual service audits. To a large extent, the M&E has been sufficient and has informed quality improvement for service delivery. Relevance, Effectiveness, Efficiency and Sustainability of the Project AMUA interventions have been consistent with local beneficiaries' requirements. For example, the underserved need delivery of quality FP/SRH services in order to manage their families. The interventions have also been consistent with country needs and especially the health sector priorities. The project has aligned itself to MoH strategy of engaging and strengthening the private sector to compliment public facilities towards the achievement of health goals, vision 2030 and MDGs. The project s strategy of franchising private providers has been relevant since it is largely demand driven and focused on enhancing access to affordable FP/SRH services for the less privileged. Capacity building initiatives have been both need and demand driven; whereas training is a crucial component in social franchising, sometimes franchisees request for it. The evaluation determined that the project has been effective and targets have been achieved in terms of key performance indicators, such as CYPs, clients or service numbers, quality assurance, donor-funds attraction, client satisfaction, skills-building and recognition by MoH/DRH. Over 576,223 CYPs were realized through the KFW funded AMUA network component. These, according to MSI s Impact Calculator, have averted over 170,000 unplanned pregnancies, over 100,000 unintended births, over 600 maternal deaths, and over 30,000 unsafe abortions. Thus, AMUA has made a difference in preventing adverse health and development outcomes through the provision of FP and related SRH services to women. Regarding efficiency, AMUA project has employed a fractional franchise model to increase availability of quality SRH services to underserved populations. The model has generally worked well and has made a difference in the way the private providers offer their services. Information from the evaluation indicates value addition on services. Information from the franchisees indicate that they have been and continue being comfortable with the AMUA processes. The quality and timeliness of inputs is by and large good and most activities are conducted as per timelines. Generally, resources have been utilized according to planned activities; however, there are areas that can be improved on to make the delivery of interventions more cost-effective and timely. Factors compounding efficiency include inadequate technical backstopping for local partners occasioned by inadequate staffing; inadequate integrated support supervision involving MOH at decentralized levels; and inadequate support for and participation in stakeholder meetings at decentralized levels. Regarding sustainability, most of the components of AMUA interventions have inbuilt sustainability mechanisms that will assure continued access to the benefits of AMUA model. These include engagement with private providers, training of CHWs, engaging with government structures and AMUA Project End Term Evaluation Report (April 2013) Page 12 of 56

13 support for coordination, and networking activities. So far, MSK has expanded the wider AMUA network through a number of grants. In addition to the 167 social franchise members supported by KfW funding, 133 additional private providers in areas like Nairobi, Garissa, and Coast have been added to the network increasing the total number of AMUA network health facilities to 280. Challenges and Lessons A number of challenges slowed down the journey towards faster realization of AMUA project results. These include competition and implications of other social franchises in the region, especially the PSI implemented Tunza network, which at temporarily resulted in cross membership by some few engaged health facilities. Commodity insecurity, affecting both contraceptives as well as HIV& AIDS, has also been a challenge and affected provision of integrated services. Other challenges revolved around managing a partial franchise especially in terms of setting service prices and retaining the already recruited members; coverage of expansive areas by staff to provide technical support and assure quality; and inadequate promotion of visibility. Despite the competition and changing environment, the project still made an edge. For example, it achieved most of the targets in CYPs, service numbers/clients, and scored highly in QTA. Regarding learning, a number of lessons have been observed. Technical capacity strengthening of private providers using a mentorship approach, as has been done is the most since it allows for application of acquired knowledge and skills, and review in subsequent meetings. Further it has been noted that membership in a social franchise network alone does not necessarily result in capacity to deliver technical interventions targeting enhanced access to FP methods; a blend accommodating network membership and sustained attitudinal transformation amongst providers would ensure better performance. Similarly, integration of services can be realized even with minimal staffing provided other inputs such as commodities and adequate training are availed to the existing providers. In addition, it is noteworthy that designing a social franchise brand is necessary but that alone is not sufficient; there is need for robust marketing strategy if the brand is to yield better results insofar as access to health services for the needy members of the community is concerned. Recommendations 1. Despite the marketing strategies put in place to promote AMUA, there is need for more innovative strategies that can ensure satisfaction and retention of clients so that they do not have reasons to move to other networks offering similar services, and recruitment of new clients. Application of behaviour change model in influencing the use of SF among the poor women who still have a lot of misconceptions on FP services is critical and should be seriously considered. There is also need for robust engagement and support for CHWs. 2. There is need for all inclusive strategies to address commodity security challenges. Apart from RH/FP commodities, there is also need for greater advocacy from the franchisor for full support of the clinics by the MOH in terms of HIV testing commodities and ARV therapy, for all clinics to be provided with Master Facility List (MFL) registration. AMUA Project End Term Evaluation Report (April 2013) Page 13 of 56

14 3. The next phase of the project should consider robust ways of integrating youth needs in its service provision for improved youth friendly services. Further, there would be need for enhanced capacity building in technical areas especially RH and HIV&AIDS service provision. 4. The strength of AMUA lies in the range of franchised services that directly reflects Kenyan government policy of FP/RH/HIV integration. This is a great opportunity for a private sector partnership to demonstrate close alignment with national health goals. There is need for proactive strategies, including influence in documenting manuals and policy documents for social franchising private health care providers. MSK can take lead in facilitating the government to champion this agenda. 5. There is need for greater PPP strategy, greater collaborations with like-minded organizations, better demand side financing, and buy in for corporate sector support. To enhance sustainability, the employment of different financing mixes will be necessary. SF requires multifaceted financing approaches that are geared towards realizing costcontainment, and maximizing utilization of services by clients across income quintiles. The AMUA project may consider a number of financing approaches including direct government input, grants and subsidies, social Insurances, and Output Based Aid (OBA). 6. In an expanded phase, focus should include facilitating other partners to adopt and scale up some of the best practices that have emerged from AMUA project. 7. In addition, focus should be on strengthening the linkage between the social franchisees with the MOH community strategy structures. The community strategy is designed to improve access to health services amongst the underserved populations and systematic engagement between private providers and CHWs is likely to influence the uptake of FP service. AMUA Project End Term Evaluation Report (April 2013) Page 14 of 56

15 1.0 Introduction 1.1 Access to Sexual and Reproductive Health Services In Kenya, the unmet need for FP is at 25% which creates a gap between the actual and desired family size 1. This means that couples are having more children than they had initially intended to due to lack of access to modern FP methods. As many as 17% of births in Kenyan are reported as unwanted or unplanned, while 25% are mistimed or wanted later 2. In Nyanza, the Ministry of Public Health and Sanitation and Ministry of Medical Services have established a provincial task force to spearhead the implementation of the National Reproductive Health (RH) policies and strategies in Nyanza Province. This is aimed at steering the province towards the achievement of the national and global targets set for reproductive health indicators. Despite the concerted efforts, it still falls short of achieving both the national and global targets. Currently skilled birth attendance is at 46 %, as compared to the MDG target of 90 %; neonatal mortality is 39/1000 whereas the target is 11/1000 by the year 2015; the unmet need for FP is at 30 % (KDHS 2008). The scenario is a replica of Western and Rift Valley provinces with unmet need for FP at 13.9% and 13.7% respectively (KDHS 2008). The demand for SRH services exceed the supply by the government, hence the need for the private sector to augment the deficit through Public Private Partnership (PPP) and other mechanisms such as social franchise, with a view to expanding the range of FP choices for women of reproductive age, engaging the private sector to increase access to quality FP and other health services that address key health priorities identified by the Government of Kenya through Ministry of Health and increasing demand for FP and other health services. Whereas its noticeable contribution of the private sectors to support the public sectors, the inadequacy of public infrastructures and HRH at all level to fast track this noble remain impediment to this realization. 1.2 Marie Stopes Kenya Country Response Founded in 1985 as an affiliate of Marie Stopes International (MSI), Marie Stopes Kenya (MSK) has been actively operational in Kenya for more than 25 years, and has aligned itself strategically within the SRH sector. Its tenure in the health sector has seen the development of tactical relations with SRH champions within Ministry of Health. Its reputation in service delivery has allowed a close working relationship with key SRH stakeholders in the country, including government structures and non-governmental partners. MSK currently manages 23 static clinics, 3 obstetric hospitals, 15 outreach teams and works with 280 social franchisees, supporting provision of a wide range of SRH Services. Service provision through MSK s established service delivery channels are also supported through various projects funded by diverse donor organizations and foundations including the Global Fund; KFW; Australian Government; 1 Kenya National Bureau of Statistics (KNBS) and ICF Macro Kenya Demographic and Health Survey Calverton, Maryland: KNBS and ICF Macro. 2 Ibid AMUA Project End Term Evaluation Report (April 2013) Page 15 of 56

16 European Commission; Bill and Melinda Gates and Packard Foundation, among others. MSK is the country s largest specialized SRH and FP organization and aims at expanding healthcare equity focusing on increasing access to and uptake of SRH and FP services among the underserved populations including youth, persons with disability, rural populations and the urban poor. 1.3 The AMUA Project Since 2004, MSK has been implementing a project branded AMUA, a Swahili word that means decide. This project is supported by a German based health sector support agency, KFW, and has been implemented in partnership with the MOH 3. The goal of the AMUA project has been to reverse the downward trends in RH and HIV/AIDS indicators as prioritized in the National Health Sector Strategic Plan II ( ). The purpose of the project has been to expand and extend AMUA social franchise network to increase access to and utilization of high-quality RH and HIV&AIDS testing and preventative services, counseling and information in an efficient and effective service delivery approach as prioritized in the National RH Policy. The project focus has been to scale up access to and utilization of RH and HIV/AIDS services in three underserved provinces of Kenya (Rift Valley, Nyanza and Western) through scale up of private service provider activities under the social franchising model of healthcare financing. With funding from other development partners such as Bill and Melinda Gates foundation, the project coverage has expanded to other regions including Nairobi, Coast, and arid areas such as Lodwar, Garissa and Kakuma Refugee Camp The project aimed at bringing together a diverse range of public (central, provincial and district MoH) and private sector health providers in a public-private-partnership (PPP) model, to increase the coordination and collaboration of all agencies. At the time of evaluation, the AMUA social franchise network component funded by KFW was working with 167 private healthcare providers (clinics and nursing homes) in 5 regions including Nakuru; Kisumu; Homabay; Kitale and Kisii all in Rift Valley, Nyanza and Western provinces of Kenya. There is an almost even distribution of the providers across the 5 regions. 1.4 Objectives of the Evaluation The overall objective of the end term evaluation was to assess the relevance of the AMUA project, its effectiveness, efficiency, impact and sustainability. The specific evaluation questions revolved around the following: To what extent are the objectives of AMUA project consistent with local beneficiaries' requirements, country needs, health sector priorities and partners' policies? To what extent have the general objective, specific objectives and expected results been achieved by the project? 3 AMUA program proposal AMUA Project End Term Evaluation Report (April 2013) Page 16 of 56

17 How well have the AMUA project activities transformed the available resources into the intended results in terms of quantity, quality and timeliness? How effectively have the stakeholders been engaged in the implementation of the AMUA project? What is the likelihood that the achievements in the various outputs/results are being sustained beyond 2012? What opportunities exist with the coming of the devolution system / county government system in Kenya? What lessons have been learnt that can be replicated or scaled up in similar projects in future? AMUA Project End Term Evaluation Report (April 2013) Page 17 of 56

18 2.0 Methodology 2.1 Study Design and Methods This was a cross-sectional participatory evaluation study involving the consultants, MSK staff and AMUA network social franchisees. It employed both quantitative and qualitative methodologies. The evaluation was conducted in two phases: 1) Review of program documents and literature review to among others inform the development of evaluation tools; and 2) quantitative and qualitative data collection in the field using semi-structured questionnaires and discussion guides. The purpose of the study was to determine whether the project had achieved its intended objectives, effectively, efficiently and realized sustainable outcomes and impacts. Furthermore, the project evaluation was expected to provide recommendations for future similar programmes The quantitative data results described how much and what had been achieved, while the qualitative data informed the perceptions and feelings of the project beneficiaries on the outcome and impact of the project Study population The study population included project beneficiaries such as women of reproductive age (15-49 years), their households, males, Ministry of Health staff, and other partners that had been involved in the program. The unit of analysis was a household with a woman of reproductive age Sampling Method and Sample Size Determination Sampling Procedure Cluster sampling was used to determine the administrative units to be involved in the evaluation. Multistage seven-tier sampling was applied. All the providers and catchment areas were listed from each project area. In each division, all the locations and the sub-locations were listed and all the villages were also listed in each sub-location, (i.e. Province-District-Division-Location-sublocation-Village-Household). The administrative units were randomly selected for the evaluation by tier. The Villages formed the cluster units from which respondents were drawn. A list of population frame by clinics and nursing homes developed from which all the samples were randomly selected for interviews Household interview Sample Size Determination The World Health Organization (WHO) formula of 30 by 10 cluster sampling was used where the five regions were grouped into two provinces to constitute two clusters - Rift Valley comprised of Nakuru and Kitale as one cluster, and Nyanza Province comprised of Kisumu, Homabay, and AMUA Project End Term Evaluation Report (April 2013) Page 18 of 56

19 Kisii as the other cluster. Using the formula, a total of 600 (300 x 2) units were earmarked as the sample for households with women of reproductive age who have used the AMUA project intervention services. For Rift Valley Cluster, each region had sample size of 150 i.e. Nakuru 150 and Kitale- 150 giving total of 300. In order to ensure that there were even representations, the sample size for Nyanza cluster was increased by 50 per site so that Kisumu, Homabay and Kisii will each had 150 thus making a subtotal of 450. In total, the study used a sample size of 750 households. The additional 150 of the calculated sample size of 600 represented any attrition, decline and absenteeism and is within the acceptable range of epidemiological buffer of 25%. Structured questionnaires were administered to all the sampled households in each region. The aim of household interviews was to collect information on the quality of services provided, relevance to the prevailing contexts, intervention strategies implemented, cost-effectiveness, impact, perceived impact of activities; success stories and the achievements, gaps and limitations of the activities as well as existing opportunities for action. Whilst the MSK AMUA project in the whole country is implemented with 280 health facilities, the component funded by KFW is implemented with 167 private healthcare providers (clinics and nursing homes) in 5 regions including Nakuru; Kisumu; Homabay; Kitale and Kisii all in Rift Valley, Nyanza and Western provinces of Kenya. There was an almost even distribution of the providers across the 5 regions. Health facilities were purposively sampled during the training and at the time of data collection with guidance of MSK staffs. Given that the numbers of health facilities were somewhat evenly distributed, the sample sizes were equal, 5 for each region. See Table 1 below. Table 1: Sample size allocations Proportionate Sample size Allocation Sample both provider & households in the catchment Nakuru Kisumu Homabay Kitale Kisii 5 Total Private health care providers (clinics) Private health care providers (Nursing homes) Households/clients from the catchment regions Focus Group Discussion with women of reproductive age using FP and men who were partners of female users (2FGD per region) Key informant Interviews (3 per region, i.e. 1 each from clinic and nursing home and one from the MOH at provincial or District level AMUA Project End Term Evaluation Report (April 2013) Page 19 of 56

20 Table 2: Sampled health facilities Kisumu Region 1. Goodwill Clinic Nyalenda 2. Mamboleo Clinic Mamboleo 3. Mumias Clinic - Maseno 4. Alpha Clinic Vihiga 5. Jamii Clinic Kisumu Kitale Region 1. Zion Clinic - Kitale 2. Highway Clinic Webuye 3. Kamutende Clinic Bungoma/Kimilili 4. Kakunga Clinic Kabras 5. Maili Saba Clinic Kitale- Kapenguria Road Homabay Region 1. Ojele Clinic 2. Rongo Royal - Rongo 3. Ranen Dispensary 4. Sony Medical Centre 5. Ogande Kisii Region 1. Dangai Medical Centre 2. Egetuki Clinic 3. Mugonga Clinic 4. Framo Medical Centre 5. Borabu Clinic Nakuru Region 1. Matutu Clinic 2. Midtown Clinic 3. Sisters Medical Centre 4. Star Clinic 5. St. Monica Clinic Figure 1: Distribution of KfW supported AMUA health facilities AMUA Project End Term Evaluation Report (April 2013) Page 20 of 56

21 2.1.3 Household Interview Structured questionnaires were meant to be administered to 750 randomly sampled households; however, the actual respondents interviewed were 747 since 3 were not found at the time of data collection. Only one respondent (female user was interviewed per household. The population frame was developed by MSK site staff at the different evaluation sites from which sampled households were randomly selected for interview. The first household was chosen using table of random numbers. The aim of household interviews was to collect information on the quality of services provided, relevance to the prevailing contexts, intervention strategies implemented, perceived impact of activities; success stories and the achievements, gaps and limitations of the activities as well as existing opportunities for action in the SRH and health care financing Key Informant Interviews The aim of Key Informant Interviews (KIIs) was to collect information on whether the project had achieved sustainable outcomes and impacts. A total of thirty eight (38) KII s were conducted, 5 KIIs with the heads of nursing homes in the 5 regions, 5 with the heads of clinics purposively selected, 5 with government staff, 6 MOH partners and 17 with MSK staff Focused Group Discussions Focused Group Discussion (FGD) guides were used to facilitate discussions and gather information in the FGDs, and a total of 10 FGDs were conducted. The discussions explored views regarding the quality of service provided by the program and solicited recommendations for improvement. Participants were selected purposively Observation Through the use of a checklist, on-site observations were conducted at various nursing homes and clinics to ascertain some of the critical equipment available for provision of FP services, cleanliness and color branding. Permission was sought from the various in charges before conducting the observations Most Significant Change (MSC) Technique The Most Significant Change (MSC) technique, which is a qualitative and participatory technique, was used to collect a selection of stories of reported changes from project activities. It was used because it is a good method for identifying unexpected changes and helps in identifying the values that are shared. The spheres of change that were focused on included changes at the individual level, changes at the community level, changes at the institutional level, and lessons learned. Data were presented in terms of the picture before and after. AMUA Project End Term Evaluation Report (April 2013) Page 21 of 56

22 2.2 Methods of Data Management Enumerators Selection and Training Enumerators were recruited from the communities of the evaluation and trained to ensure quality data. They were trained on, knowledge and skills on interviews at the households, filling in responses in the questionnaires, selection of first household to be interviewed and their roles and responsibilities during the evaluation period. The criteria for selection included possession of at least secondary level certificate of education and fluency in both English and local language. The participatory training included simulation of data collection exercise amongst the enumerators in order to identify some of the challenges expected and also to assess the level of success of the training. The successful trainees were considered for data collection exercise with supervision to ensure data quality assurance and consistency. For qualitative data collection, interviewers were identified. They had a minimum of university education and had conducted at least 2 FGDs prior to this evaluation. They were trained on qualitative data collection techniques to enable them capture issues as they came from the respondents Involvement of Marie Stopes Kenya Marie Stopes Kenya was involved from the beginning of the evaluation to be fully inclusive of the major stakeholders and for easy logistical management in the field Pre-testing of Data Collection Tools The household questionnaires were translated and pre-tested. The pre-testing was meant to serve two purposes: the primary purpose was to focus on the content of the questionnaire to ensure reliability and practicability of the instrument, and the second purpose focused on the average time taken and the capability of the enumerator to administer the questionnaire. This informed logistical arrangements including the desired number of interviewers and supervisors. 2.3 Mobilization of the Communities for the Evaluation Exercise Community structures, project staff and provincial administration were effectively and proactively involved to support in the sensitization of targeted communities about the evaluation. Authority letters were written and issued to enumerators for ease of identification, and also to erase any doubt and suspicion that would obtain. This heralded acceptance by all the participants. 2.4 Ethical Considerations Informed consent was obtained from clients before any interviews were conducted. Participants were informed about the benefits of participating in the study and that they had freedom to AMUA Project End Term Evaluation Report (April 2013) Page 22 of 56

23 participate, or opt out at any stage of the interview. It was indicated that the findings and recommendations from the study would inform the development of future SHR programs in the region/country. All the data (electronic copies, data forms and key informant notes) have been securely stored. Confidentiality: Confidentiality of all data collected from clinics, nursing homes and community were maintained and guaranteed. Respondents were assured of the confidentiality of their responses, and no direct identifiers were obtained. 2.5 Data cleaning Data were cleaned in two phases. The first cleaning was executed in the field where the completed survey questionnaires were reviewed each day. The completed questionnaires were checked for completeness, clarity and correct coding. The second phase of cleaning was conducted after the quantitative data had been entered in the computer Data analysis The quantitative data was analyzed using Statistical Package for Social Scientists (SPSS) software (version 17.0, Chicago, Illinois, USA). After analysis, the data was then exported to Microsoft Excel in order to draw tables and figures. The qualitative data from KIIs and FGDs was analyzed manually, and included coding, summarizing, categorizing, direct quoting, and comparisons by sub-themes and themes. AMUA Project End Term Evaluation Report (April 2013) Page 23 of 56

24 3.0 Findings and Discussions In this report, findings are presented along thematic areas central to the project namely achievement of specific project outputs, relevance, effectiveness, efficiency, impact and sustainability. Under the section on achievement of project outputs, the areas covered are access and utilization of RH and HIV&AIDS services, behaviour change and demand creation, capacity of private providers to offer services, coordination and collaboration between the public and private sector service providers, and monitoring and information sharing. 3.1 Demographic Characteristics of the Respondents At the onset of the evaluation, the target respondents for the quantitative component of the evaluation were 750. However, the actual number interviewed during the evaluation was 747 giving a response rate of 99.6%. Seven declined to state their gender, and the rest were female (n=734; 99%). The distribution of respondents by region/sites is shown in figure 2 below. Figure 2: Distribution of respondents by region/site N= 747 The majority of those interviewed were between ages years old, and over 70% of those interviewed were married. In terms of education, a large proportion of those interviewed ranged from those who had some primary training to those who completed secondary school education. Business was the main source of income for a good proportion of interviewees (48.6%), with those practicing farming and those without a source of income being approximately 17% each. See Table 3 below AMUA Project End Term Evaluation Report (April 2013) Page 24 of 56

25 Table 3: Socio-demographic characteristics of the clients Characteristics Frequency % Age Group N=734 < 18 Yrs Marital Status N=742 Currently Married Not Married living with partner Widowed Divorced Separated Single/not in union Highest Education N=745 No Education 15 2 Some Primary Primary completed Some Secondary Secondary Completed Vocational Schooling Some University/college 37 5 Completed University/college Main Occupation N=722 None Farming Business Salaried Others Missing Access and Utilization of RH and HIV&AIDS Services One of the envisaged results of the AMUA project was the realization of scaled up access and utilization of RH and HIV/AIDS services and counseling to target groups through an expanded social franchise network. AMUA Project End Term Evaluation Report (April 2013) Page 25 of 56

26 In Kenya, health care services are offered through a complimentary approach involving the government, as well as Civil Society Organizations (CSOs) that include Faith Based Organizations (FBOs), Non-Governmental Organizations (NGOs), and the private sector. The GoK has a dedicated agency for receiving and distributing medical supplies, including FP commodities. This agency is known as the Kenya Medical and Supplies Agency (KEMSA) and provides medical supplies to all public facilities, as well as many NGO and private facilities. Public health care services are generally free at the point of care. The majority of national health coverage is paid for through tax revenues with donor funds also contributing a significant proportion for public health budget expenditures. The private health sector is wide-spread in Kenya and used by clients of all socio-economic levels. The government acknowledges publicprivate partnerships and appreciates the significant role played by the private sector Utilization of AMUA Services The end term evaluation determined that all AMUA services such as long term methods of FP are available in many AMUA facilities but not all. Even though FGD participants indicated that some of the services offered by AMUA facilities are also offered from other private clinics, chemists, and government hospitals, they observed that AMUA clinics are accessible, the services are good with short waiting time, and have friendly service providers with good counseling sessions. The evaluation determined that FP services offered in most AMUA health facilities include FP counseling and provision of FP methods such as injectables, pills, IUCD, Jadelle, and condoms. As such, AMUA clients use varied FP methods with injectables being the most widely used (28%) while male sterilization is the least used method (0.3%) amongst the clients; see Figure 3 below. AMUA Project End Term Evaluation Report (April 2013) Page 26 of 56

27 Marie Stopes Kenya s AMUA project is well recognized as a source of FP with the majority of clients (n=448; 60%) depending on it for their current FP needs. See Table 4. One of the reasons for using AMUA clinics is the quality of service. A franchisee in Kitale reported that as part of quality assurance they normally offer counseling (mandatory to all clients) before they are given any method. Additionally, the service providers are friendly and clients walk into the facilities with confidence. Table 4: Source of current FP methods FP method Source N=634 Frequency Percent GoK/public facilities Marie Stopes/ AMUA Project Private Health facilities Pharmacy/drugstore Others More than half of clients (n=448, 60%) received services direct from AMUA project (See Table 4 above). The majority (n=639, 86%) felt that the current available FP/RH services met the needs of the community. Some clients raised concerns about poor clients inability to pay, despite of the fact that over 80% of the clients were able to afford services (see Figure 4 below). Figure 4: Cost rating by clients for FP/RH services in AMUA clinics AMUA Project End Term Evaluation Report (April 2013) Page 27 of 56

28 The integration of FP with other services, especially HIV&AIDS services, was also examined as part of this evaluation. Initially, the focus of AMUA (phase I) was on the provision of long term FP methods. Phase II has included integration of other services such as HIV/AIDS services, MPAC and also youth friendly FP services. In order to realize integration in the clinics, a strong network for referral has been put in place. To track integration, there is a comprehensive reporting tool that is used to track services offered to clients. Despite these efforts, there are some challenges, especially with provision of HIV/AIDS services due to erratic commodity supply for the latter. We have been able to integrate well FP with HIV/AIDS services though we also have challenges of erratic supply of HIV kits, FP commodities etc. and this leads to me having depo to give a mother but I cannot do HTC to the client which means the integration now has not taken place. If the commodities can be supplied equitably that will help A provider in Kitale Commodity supply was also reported to be an issue affecting cervical cancer screening: We have been doing cervical cancer screening but we are now running short of commodities. Supply is an issue that is beyond the district; it is at the ministry level for instance yesterday we received commodities that are expiring in the next two weeks. What will we do with these drugs and the disposal procedure is still not well stipulated in the policy. You realize they bring them late in the evening when we have gone home and so the security keeps them Figure 5: HIV services offered within AMUA network health facilities Almost all clients (n=603, 90%) confirmed that service integration has been achieved in AMUA clinics and that multiple services are now offered in the same rooms. When asked what services were offered, they rated the services as shown in Figure 6 below: AMUA Project End Term Evaluation Report (April 2013) Page 28 of 56

29 Figure 6: AMUA Clients Perception of Integration in Clinics When asked their opinion on AMUA service provision, clients rated each of the attributes as indicated in Figure 7 below, with the majority describing services as being affordable with friendly and kind staff. Figure 7: Clients Opinion about AMUA Services AMUA Project End Term Evaluation Report (April 2013) Page 29 of 56

30 Figure 8: Last time client visited AMUA facility Regarding methods of payment, almost all the clients (98%) made payment in cash when they visited the AMUA network health facilities for RH/FP services. Reliance on cash payment may at times make it difficult for clients to access RH/FP services, particularly when services are needed yet there is no money at hand. Further, given that the propensity to pay health care services is low among the poor, cash payment for services that are largely perceived to be precautionary may not be prioritized by vulnerable populations like those targeted by the project. Therefore, use of voucher system to pay for health care (FP) could assure better access to RH/FP services. Regarding transport cost, over 70% of the clients seeking care at AMUA clinics spend less than 51 Kenya shillings on transport to access the facility with less than 10% spending more than 100 Kenya shillings. See Figure 9 below. Figure 9: Transport cost to AMUA facility AMUA Project End Term Evaluation Report (April 2013) Page 30 of 56

31 The evaluation assessed waiting time in AMUA health facilities. Those who sought care at the various AMUA clinics appreciated service availability and waiting time with approximately 90% of the clients receiving care within 30 minutes after arrival during their last clinic visit. Almost all clients (n=719, 99%) got the services they required, with over 94% (n=677) indicating that they received drugs in their last clinic visit. Table 5: Services Obtained During Last Visit Characteristics Indicators Frequency % Got the service/s you wanted in last visit Yes No 2 0 Got the drugs you wanted in last visit Yes No 23 3 Not application 18 3 Reasons for not getting drugs on last visit Stock out Did not have money 3 14 Others specify 9 41 Are the poor seeking services as advocated by AMUA Yes No Regarding ability to pay for services, all AMUA clients who sought services in the previous 1 year prior to the evaluation reported that they were able to pay for the services they sought. Table 6 below shows services received and clients ability to pay. Table 6: Service received at last visit to AMUA facility and ability to pay for the service Less 1 Month 1 to 3 Month 4-6 month 7 M to 1 year ago Able to Not able Able to Not able Able to Not able Able to Not able pay to pay pay to pay pay to pay pay to pay Condom distribution Implants MPAC/PPH IUCDs FP Counseling Pills Cervical Cancer Injectables / DMPA VCT/PITC Safe Deliveries PMTCT BTL Referred STI Treatment Vasectomy Referred AMUA Project End Term Evaluation Report (April 2013) Page 31 of 56

32 Regarding linkages between MSK and AMUA clinics for provision of RH services, the majority of the respondents (97%) rated the linkages as being good and that they made access to quality RH/FP services reliable. And as indicated in Table 7 below, the majority of the AMUA clients in Kisii, Kitale, Nakuru and Homabay at the time of the evaluation were found to be mostly using injectables, way above even condoms, in all the evaluation sites. Kisumu had the least in both uptake of injectables and condoms Table 7: Comparison between utilization of Condoms and Injectables by AMUA clients in different sites Sites Condoms (%) Injectables (%) Kisumu 0 1 Kisii Kitale 8 57 Nakuru 5 41 Homabay 5 43 AMUA Project End Term Evaluation Report (April 2013) Page 32 of 56

33 Health Facility Case study Framo Clinic, Borabu Maternity and Nursing Home, and Daraja Medical Centre The three AMUA clinics in Kisii region have strong linkages and involvement in other government initiatives. For example, both Daraja Medical Centre and Borabu Maternity and Nursing Home are linked to the area Village Health Committees (VHC) and the Community Health Communities (CHC), have Health Facility Communities (HFC), and participate in both divisional and district stakeholders forums. All the three facilities have Community Health Workers (CHW) and are linked to Community Units (CU) For services, the health facilities have Ante Natal Care (ANC), immunization, FP, deliveries, and HIV Testing and counseling (HTC) services which are offered from Monday to Friday and brief service charters are well displayed. Except for Daraja where there are no ARVs, PMTCT services are also offered. On health information management (HIS), all the service data are available and displayed, for example data for ANC, immunization, FP, skilled delivery, neonatal deaths and exclusive breastfeeding, The facilities have a number of registers/forms including MOH 512 used for FP, MOH 362 for HTC, MOH 301 for inpatient, MOH 510 for immunization, MOH 405 for ANC registration, MOH 204 for outpatient register, MOH 511 used by NASCOP, MOH 704 used as a tally sheet, MOH 705 summary sheet, MOH 701 outpatient, MOH 105 for service delivery, MOH 731 for HIV, and MOH 406 for postnatal. Summaries are done using MOH 711. Linkages with the community: There are referral mechanisms in place with other health facilities as well as with the community. Dialogue days are conducted in the health facility. There are support groups, as was the case in Daraja Medical Centre, and these are complemented with household visits. The support groups linked to the facility included Daraja United Women Group with 10 members and focused on VCT, FP, HIV/AIDS and care; Hope For Survival- Post Test Group with 30 members and focused on stigma reduction; and Usafi Youth Group with 9 members and focused on HIV behavior change; Infrastructure and client accommodation; In all the three facilities there are adequate seating provisions for clients, covered waiting rooms, designated sections for MCH/FP, examination/ counseling rooms with privacy, safe water points, functional latrines, functional waste disposal pit/ area, information education and communication (IEC) IEC job aids displayed on subjects such as STI/HIV/AIDS, VCT, PMTCT and HBC; counseling guidelines, cards/ pamphlets - some in local language. Equipment and commodities: The facilities have arrange of equipment and commodities including autoclaves and FP kits with kidney dishes, dalli pot, tenaculum, sponge holding forceps, speculum, uterine sound. Supervision: The facilities have benefitted from supervision visits from the district team and records show supervision focus on among others on ANC, immunization, FP, delivery, STI/HIV/AIDS and VCT. Achievements: Since joining AMUA network, the facilities have realized improvement in their facilities and increase in uptake of services such as FP, HTC, Immunization and PMTCT. Commodity security has improved and there is improved networking and engagement with MSK and CHWs, and there has been increase in income. Apart from joining the AMUA network, other success factors are health education, use of CHWs, good rapport (public relation), being client driven client satisfaction, community involvement, proper client mobilization, and capacity building on RH by MSK. Lessons learnt: Working with the communities can improve service uptake Funding is essential for service delivery Networking is important because it enables sharing of information It is proper to be open up and learn more from others One should not underestimate oneself AMUA Project End Term Evaluation Report (April 2013) Page 33 of 56

34 3.2.2 Family Planning Commodity Security The FP commodities security is essential for quality service delivery. Most of the AMUA network facilities sampled had a commodity management plan. The main source of FP commodities was indicated to be government (KEMSA). Availability of FP commodities has been assured in the nearby private facilities than in the government ones. Logistical challenges have occasionally affected service delivery though such cases were reported to be few. Whenever commodity stock outs occurred clients would be referred to a nearby health facility. The orders for replenishment are usually placed using phone calls, s and normal reporting documents. Another concern is the short expiry date for some of the commodities delivered - some expire after only two weeks. Other challenges include the fact that orders are made on specific days, and deliveries, which are made directly to the facilities, are uncertain and unreliable because the supplies are sometimes less than what was requested. Apart from government supplies, providers also procure commodities from pharmacies when there are stock outs and at times they receive form MSK AMUA project. To address commodity security challenges, the District Reproductive Health Coordinators (DRHC) track commodity usage by among others compiling monthly reports to determine stock availability and use the information to re-distribute commodities to needy health facilities. The procurement of commodities is based on the pull principle, whereby health facilities request for commodities based on utilization rates, as opposed to a push system where commodities are routinely distributed irrespective of utilization trends. Facilities closer to major government health facilities like Nakuru and Kisumu tend to be disadvantaged since the district stores also serve them Adherence to Government Policy Guidelines and Protocols Policy is one of the key factors that need to be given keen attention in promoting service delivery. The policy provisions have affected a number of things including access to FP commodities, service provision procedures, and friendliness of the service provision environment. The decentralized health systems contained in policy frameworks have generally allowed private providers to give FP services and move the services close to people. There are technical guidelines, protocols, standard operating procedures, and client rights in place and are being followed by the health care providers. The foregoing notwithstanding, there is need to review existing policies to better address the challenges related to commodity security particularly at the county government within the ambit of efficiency concerns. Generally, as envisaged during its design, AMUA project has increased access to quality and affordable FP services in the five focus regions and there are concerted efforts to deliver integrated services to clients. Optimal realization of the objective and especially integration has been affected by commodity insecurity for FP and other services, as well as inadequate staffing at health facility level. The clients are generally happy with services save for cost which is AMUA Project End Term Evaluation Report (April 2013) Page 34 of 56

35 sometimes compared to rates offered by facilities in other social franchise networks such as Tunza, Going forward, there is need to make significant provisions for strengthening commodity security for FP and HIV&AIDS related services, and enhanced support to government structures for supervision and coordination. 3.3 Behaviour Change and Demand for RH and HIV&AIDS Services The second envisaged result of AMUA project was improvement in awareness of reproductive health issues including FP, positive behaviour change and increased demand for RH and HIV&AIDS services and counseling. Generally, AMUA project has provided information to the public regarding SRH services offered at the various outlets through printing and distribution of IEC/BCC posters, brochures and posters developed and endorsed by the DRH, and through radio spots, talk shows on FP/SRH and one-on-one information sharing. In the course of implementing the project, the project undertook a review of its demand generation strategy. Based on this review, the project embarked on an enhanced generation strategy which involved the development of specific messages and targeting strategies to reach the AMUA clients namely the poor who can only afford to pay low, subsidized fees. The strategy also entailed review and reinforcement of an agreed pricing structure for franchise supported services. Specifically, AMUA project demand generation objectives focused on increasing use/adoption of all FP methods, integrating Youth Friendly Services (YFS) including reproductive health and contraceptive use within the existing services, and increasing uptake and utilization of other services such as Cervical Cancer Screening (CCS), and safe motherhood. The communication objectives included increasing positive perception towards FP /SRH; increasing positive product perception on FP methods, particularly IUCDs and implants; increasing awareness on features and benefits of AMUA FP clinics among potential clients; and increasing client flow to AMUA FP clinics. In order to influence individual level knowledge, attitude and perceptions, the Behaviour Change Communication (BCC) activities revolved around use of peer groups to educate others; engagement with the church; use of organized groups such as women groups; and information sharing at health facility waiting bays. The BCC messages delivered include the importance of FP, importance of knowing HIV status, and the importance of health facilities. The messages have been delivered using mediums such as: Interpersonal communication (CHW and peer educators); promotional campaigns for uptake of services through use of applicable communication channels such as posters, radio, and brochures. The radio has shown to be effective especially local FM radio stations, more so those with interactive question and answer sessions. There is also provision of IEC materials, such as Shujaaz pullout in a local newspaper, Saturday Nation. The pullout is comic book aimed at engaging the youth. Shujaaz is part of the well-told stories approach which is aimed at encouraging talk and debate. Promotional campaigns for uptake of services through use of applicable communication events such as road shows and enhanced of AMUA Leo events. The AMUA Leo day AMUA Project End Term Evaluation Report (April 2013) Page 35 of 56

36 concept has entailed in-reach within the health facility with selected services being subsidized. There is bundling of services, i.e. promotion one service and selling others. AMUA Leo is largely a promotional event known us experiential marketing In addition, the project has promoted the establishment of information and referral networks and worked with franchisees on strengthening local referral systems and networks. These have included service information and referrals provided through CHWs. Providers have been responsible for maintaining the CHW through provision of their motivation and monitoring performance. The project provided technical assistance in monitoring the effectiveness of CHW activities delivered at community level. Further, ensuring highly visible branding of AMUA social franchise clinics has also been a key marketing activity, and touch-up and facebranding has been Figure 10: Branded AMUA clinic undertaken in engaged clinics. The effectiveness of demand creation activities has been partly assessed through client exit interviews to determine the level of satisfaction of clients. Over 90% express satisfaction with services offered in AMUA clinics. Willingness to recommend services to others stand at over 95%. The demand creation activities have complemented each other fairly well and have yielded results with not only greater awareness for services but also increased demand for the same. The clients were seeking FP/RH services mostly after sharing with one another and after being reassured by their peers that a particular method is good. They prefer seeking FP/RH services at the private medical facilities due to a number of factors including privacy, confidentiality, convenience, cleanliness of the facilities, advertisements of the AMUA branding Those seeking FP/RH services mostly do so after sharing with one another and after being reassured that the type of a particular service is good. Of the 747 persons interviewed 99.5% (743) of the respondents admitted having ever heard of the term FP with 83.1% (621) acknowledging that they are currently on FP methods. The main reason why families use FP methods is so as to be able to space their families [(434) 58.1%] followed by the desire of not wanting more children [(136) 18.2%]. AMUA Project End Term Evaluation Report (April 2013) Page 36 of 56