Formative Evaluation Report

Transcription

1 . Formative Evaluation Report for The Project entitled Accelerating efforts to reduce maternal, neonatal and child mortality in the Northern and Upper East regions of Ghana Evaluators: Timothee GANDAHO, MD, PhD, Samuel BOSOMPRAH, MSc, PhD, September,

2 Table of Contents Acknowledgements... 5 Executive Summary... 6 Background... 6 Key Findings... 7 Conclusion... 9 Recommendations Introduction Background Specific objectives of the formative evaluation: Scope of the Evaluation and Evaluation Questions Scope of the formative evaluation Evaluation questions: Evaluation Methods Ethical considerations Study design Sample size consideration and sampling Field work/data collection Data Analysis Data archiving Quality Assurance Evaluation findings Lessons learned Conclusions and Recommendations Conclusions Recommendations References Appendices AP1. Informed Consent Form AP2. Evaluation tools (In-depth interview guides, Short questionnaires, FGD guide) AP3. Evaluation Framework AP4. List of National Decision-Makers/Stakeholders Interviewed AP5a. Evaluators work schedule AP5b. Percentage change in institutional neonatal deaths by districts in the two regions

3 Acronyms and key definitions Acronyms ANC Antenatal Care AP Appendix BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity Based Agents CHN Community Health Nurse CHO Community Health Officer CHPS Community Health Planning and Services C4D Communication for Development DA District Assembly DHIMS Health Information Management System DHMT District Health Management Team EC European Commission EMBRACE (model) European Model for Bioinformatics Research and Community Education EmONC Emergency Obstetric and Neonatal Care FGD Focus Group Discussion GHS Ghana Health Service HBPNC Home-Based Postnatal Care HIV Human Immunodeficiency Virus HRBA Human Rights-Based Approach HSS Health Systems strengthening IMNCI Integrated Management of Neonatal and Childhood Illnesses JHPIEGO Johns Hopkins Program for International Education in Gynecology and Obstetrics JICA Japan International Cooperation Agency KOICA Korea International Cooperation Agency KMC Kangaroo Mother Care LBW Low Birth Weight MAF MDG5 Acceleration Framework and Action Plan MDGs Millennium Development Goals M&E Monitoring and Evaluation MNCH Maternal Newborn and Child Health MoH Ministry of Health NGO Non-Governmental Organization NBC Newborn care NCC Newborn care corner NCU Neonatal Care Unit NMR Neonatal Mortality Rate NR Northern Region OECD/DAC Development Assistance Committee of the Economic Cooperation and Development PATH An International Health Organization PNC Post Natal Care PPME Public Private M E QI Quality Improvement QA Quality Assurance 3

4 UER UGMS UNICEF URC WHO UN UNFPA USD USAID Upper East Region University of Ghana Medical School United Nations Children s Fund University Research Co, LLC World Health Organization United Nations United Nations Population Funds United States Dollar United States Agency for International Development Key definitions Low birth weight Newborn death Preterm birth Weight of less than 2,500g, irrespective of gestational age Death within 28 days of birth of any live-born baby regardless of weight or gestational age A baby after born less than 37 completed weeks of gestation Still birth A baby born with no signs of life at, or after 28 weeks' gestation (WHO)

5 Acknowledgements The National Newborn Subcommittee members and UNICEF Ghana staff contributed to the improvement of the evaluation tools, framework, preliminary results and draft report with their constructive suggestions and comments; the evaluators wish to express their appreciation to them. The evaluators would particularly like to acknowledge the technical and financial assistance received from UNICEF for this evaluation. We would also like to express our sincere gratitude to the experts and national key informants from MoH, GHS, UNFPA, UNICEF Accra /Tamale, WHO, European Commission, USAID, JICA, the Embassy of Japan, USAID/HSS Project, USAID/JHPIEGO project, PATH, the Pediatric Society of Ghana, the Society of Obstetricians and Gynecologists of Ghana, the Ghana Registered Midwife Association, the Teaching Hospitals, the School of Public Health, Project Fives Alive, the University of Ghana Medical School (UGMS) and the Coalition of NGOs on Health; who provided valuable information used for this evaluation. The contributions of other United Nations agencies and development partners who were key informants in this evaluation process are also greatly appreciated. Special thanks to all national, regional and district Leaders or Directors or in-charge and newborn care focal persons of Ghana Health Service HQ/NR/UER who were interviewed at national, regional and district levels. The information they have provided was key to this evaluation. The evaluators are very grateful to the highly motivated health providers, especially the community health officers, the community health nurses and midwives of the Northern region and Upper East region, and to all the mothers who participated with enthusiasm in the focus group discussion sessions as beneficiaries of newborn care. They have generated critical information for this formative evaluation for the newborn care project in the Northern and Upper East regions. Furthermore, we would like to commend the technical support and the valuable and relevant comments received at various stages of this evaluation from UNICEF Staff Dr. Hari Krishna Banskota, Dr. Victor Ngongalah, Dr. Imran Ravji, Mrs. Felicia Mahama, Mrs. Anna Maria Levi, and Mr. Clemens Gros and from GHS Staff Dr. Isabella Sagoe-Moses and Dr. Cynthia Bannerman. We also extend our appreciation to all who contributed directly or indirectly to this evaluation. 5

6 Executive Summary Background The death of infants in the first 28 days of life is increasingly becoming a global health concern. This is especially relevant as it undermines the achievement of the millennium development goal: to reduce under-five mortality. Many countries in the developing world have, over the years, implemented a series of interventions to reduce the burden of under-five mortality. In line with the National Child Health Policy, UNICEF with funding support from the Government of Japan, has been providing technical and financial assistance to GHS at the National level and in a selection of fourteen districts of the Northern and Upper East regions of Ghana since October 2011 in order to implement the project entitled Accelerating efforts to reduce maternal, neonatal and child mortality in the Northern and Upper East regions of Ghana. This project was the subject of this evaluation. The following are the key findings of the evaluation and recommendations for possible policy action. The Specific objectives of this formative evaluation are: 1. To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child survival interventions with a focus on community-based (Home-based Postnatal) Care (HBPNC) and facility-based integrated Management of Newborn and Childhood Illnesses approaches in the selected districts of the Northern and Upper East regions; 2. To ascertain the project s contribution to capacity building that is, developing training resources and facility structures to respond to the high levels of newborn morbidity and mortality in the selected districts of the Northern and Upper East regions; 3. To ascertain the effectiveness of the evidence-based advocacy by the project on the national policy environment on issues related to newborn survival; 4. To draw lessons on the implementation capacity for the national expansion of the essential newborn care model through home-based early postnatal care. The scope of the formative evaluation covered the areas of implementation of the project in the two regions (Northern and Upper East Region) of Ghana and at the national level. The evaluation covered the two phases of the project from September 2011 to December The evaluation focused on, and included the following: final beneficiaries, service providers, actors at the sub-national decision-making level (district and regional health authorities), actors at the national decisionmaking level, national professional societies and academia. The formative evaluation attempted to provide answers to a number of questions to meet the Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC) evaluation criteria as it pertains to relevance, effectiveness, efficiency, and sustainability as well as UNICEF s Coherence and Human Rights-Based Approach (HRBA) to Programming and Equity for all target groups. The formative evaluation employed a mixed method design consisting of qualitative and quantitative components. The qualitative component consisted of in-depth interviews with 12 key national decision-makers, donor partners, 16 sub-national health authorities, 12 service providers and 2 focus group discussion (FGD) sessions with mothers drawn from the communities in the two project districts. The quantitative component involved abstraction of neonatal health indicators from the District Health Information Management System (DHIMS 2) and other relevant data sources based on indicators developed from the evaluation questions as well as project-specifics to assess the project s success and effectiveness. 6

7 Key Findings The main findings are summarized below according to specific evaluation objectives: Objective 1: To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child survival interventions with a focus on community-based (Home-based Postnatal) Care (HBPNC) and facility-based integrated Management of Newborn and Childhood Illnesses approaches in the selected districts of the Northern and Upper East regions; The established level-two newborn care units in the district hospitals have contributed, to some extent, to an improved neonatal survival through an improved management of sick newborn babies. For example, institutional neonatal deaths per 1,000 live births reduced by about 51% in the Northern region and about 43% in the Upper East region over the project period (from 2012 to 2014). But challenges still remain with cases from the communities arriving late for management. However, it should be noted that the project was implemented in well identified districts and not in the entire region that contains a lot more districts; thus the findings disaggregated and analyzed by project districts show mixed results. During FGD, the beneficiaries expressed satisfaction with the home-based care for their babies received from the CHOs/CHNs. They also indicated having received basic counseling on health and wellbeing of mothers and babies and that the regular interaction with the CHOs/CHNs and community volunteers provided the opportunity to share their experience and bring up their challenges relating to the provision of care for their newborn babies and managing of their own health. Perceptions and reported statements from beneficiaries indicate that the health and wellbeing of babies have improved substantially to their satisfaction. They reported positive behavioral change and less diseases due to the newborn care intervention, especially education by nurses and volunteers during home visits and supportive communication activities. A review of the project proposal indicated that due considerations were given to changes in the burden of neonatal deaths in the selection of the project regions. It also indicated that the regions were chosen to consolidate the gains recorded following a series of interventions in the past. A review of the Medium Term Health Expenditure Plan for and the Ghana Shared Growth and Development Agenda for showed that many activities therein can directly and indirectly impact the lives of the newborn. The National Newborn Strategy provides a more focused framework on newborn survival, which can be operationalized through annual plans and budgets with the support of health partners. Objective 2: To ascertain the project s contribution to capacity building that is, developing training resources and facility structures to respond to the high levels of newborn morbidity and mortality in the selected districts of the Northern and Upper East regions; The project supported capacity building workshops for all district directors of health services and district public health nurses in the project districts on essential newborn care. They are now able to plan for newborn care activities. A review of the districts annual plans showed that they have all featured newborn care activities. Key informants reported that the project trained sub-national personnel as trainers who in turn trained sub-district and community health service providers on newborn care. 7

8 A desk review of project documents showed that a total of 4,151 health workers (369 Doctors, midwives, medical assistants, nurses, 816 CHOs and CHNs) plus 1748 community volunteers and 1218 Red Cross mother-to-mother support groups with 18 supervisors and facilitators) were trained on essential newborn care. However, while service providers reported satisfaction with the training content, the capacity created might diminish over time especially under high attrition situations. Out of $1,812,187, 45% ($808,356) were spent on training and capacity building and 35% ($617,271) spent for community and facility service delivery and demand generation. The cost of training a health professional in newborn care was $ on average per trainee. Key informants, reported that the application of skills and knowledge acquired during the newborn care trainings reinforced with supervision, monitoring and mentorship have helped to improve the quality of service delivery by the trained providers. Some of the beneficiaries reported to have been educated on essential newborn care through community durbars and information from nurses and volunteers who spent time with them during home visits. Objective 3: To ascertain the effectiveness of the project s evidence-based advocacy for issues related to newborn survival and national policy environment. The aim of the project to improve neonatal survival is very much aligned with the national child health policy. A desk review of relevant policy documents showed that the objectives of the project were aligned with the Government of Ghana s Child Health Policy. The project is very much placed within UNICEF s global mandate to improve children s health. The project is operationally aligned with UNICEF s significant presence in northern Ghana including a field office in Tamale with technical and operational staff who provide close technical and monitoring support during the project. The project is also very well aligned with the EMBRACE model articulated in the Government of Japan s Global Health Policy both conceptually as well as operationally. The project supported the processes leading up to the development and launch of the National Newborn Strategy and Action Plan. Specifically, the project supported MoH/GHS to organize 3 national level stakeholder meetings on newborn health. It also supported the development of bottleneck analysis tools and decentralized monitoring and planning on newborn health. As reported by key informants, the launch was very successful, and the advocacy and communication around it contributed to place newborn health on the national agenda. The Ministry of health and partners, recommended a newborn strategy and Ghana Health Service was tasked to have it developed with newborn indicators for performance monitoring. The national newborn strategy was launched in July The advocacy and communication around it was such that it was attended by parliamentarians, civil society, embassy representatives, some key media personnel and representatives from other sectors. In order to implement the National Newborn Care Strategy, all 10 regions now have a newborn health focal person. Objective 4: To draw lessons on the implementation capacity for national scale-up of the essential newborn care model through home-based early postnatal care. The Newborn Sub-Committee coordinates newborn care activities of all the partners involved in implementing newborn interventions. Each region and district has a focal person for newborn care. The training modules have been adopted and are being rollout to other regions and home visits are being integrated into the CHPS structure. 8

9 The DHMT/DA is supportive to newborn care but is yet to demonstrate ownership to consolidate the achievements and the expansion of the newborn health interventions within available or mobilized resources for the district. Its resources are not sufficient to be able to selfsupport the needs for a long-term sustained newborn care intervention. Challenges however remain. They pertain to the adequate mobilization of resources for such ownership and the remuneration of community-based volunteers, the building of capacity for new staffs with more training sessions, more NCU equipment and resolving problems pertaining to fuel and motorbikes supply for home visits. Government resources to the sector are limited and remain basic for the regions where the health services depend on internally generated funds mainly from the national health insurance scheme. Unfortunately, the delay in payment by the insurance scheme is further threatening service provision. Service providers admit that despite the success of the project, there exist some bottlenecks, which need to be addressed before scaling-up the essential newborn care model through homebased early postnatal care. These include lack of motorbikes, insufficient personnel due to staff attrition or trained nurses going back to school for further training, lack of means of transportation for supportive supervision or performance monitoring, and volunteer fatigue due to the absence of incentives. Conclusion The home-based postnatal newborn care and neonatal intensive care models have been effective in contributing to improved newborn survival in the two project regions of Upper East and Northern regions of Ghana, to the extent possible given the scope and reach of the intervention. The enhanced capacity of NCUs, with essential newborn care equipment as well as the enhanced capacity of health personnel in terms of skills acquired for management of sick and preterm babies, have been important enabling factors for saving the lives of many babies in the project districts. The evidence-based advocacy efforts at all levels contributed significantly to making newborn issues a national priority, especially culminating in the development and launch of a National Newborn Strategy and Action Plan. Recommendations The challenge for improving newborn health lies in ending preventable newborn deaths and securing Ghana s future. Success will be measured in terms of lives saved and lives improved. Success will depend on meeting the needs of women and their babies throughout the continuum of care and committing to the following action items: National Level: 1. The Government should commit enough resources to operationalize the National Newborn Strategy and Action Plan. The Newborn strategy could be used as a framework for donor support. Donor assistance should be mapped onto strategy priorities and donor projects have to be coordinated in order to achieve strategy objectives. A system should be put in place for the effective monitoring and assessment of achievements and resource management tools should be setup so as to ensure accountability. A national budget line for newborn activities needs to be envisioned. An advocacy group may also be put in place to ensure continued resource mobilization for Newborn Strategy implementation. 9

10 2. The MoH/GHS should ensure that every district hospital has a Newborn Care Unit for secondary (level 2) care. All regional and teaching hospitals should have Newborn Intensive care Units (NICU) for tertiary (level 3) care. Health Centers and Polyclinics where delivery is conducted should be provided with Basic Emergency Obstetric and Newborn Care (BEmONC) including Newborn Care Corners (NCC). 3. The GHS/MoH should establish resource centres in Regional and Teaching Hospitals along with NCU, to the extent possible using existing structures, to facilitate on-the-job training for newborn care. Staff from the resource centres should deliver a transferable skills program through mentorship and periodic specialists visits to lower level facilities. 4. The GHS should scale-up the home-based postnatal newborn care model to all districts in the regions and to other regions. The evaluators do not anticipate any delay if national and sub-national decision-makers commit to this course. The materials have already been developed and lessons learned can speed up a nationwide scale-up. GHS could take advantage of the fact that in all ten regions, there is ongoing newborn activities in some of the districts supported by various donors such as UNICEF (2 regions), USAID HSS (5 regions), USAID JHPIEGO (4 regions), PATH (4 regions), JICA (1 region) and KOICA (1 region). This will require coordination and harmonization on a minimum package of effective newborn care for the reduction of newborn mortality. 5. The MoH should review the curriculum of the Midwifery and Community Health Training Schools to include issues on newborn care or update and strengthen any existing ones such as the training programs developed in collaboration with UNICEF and other development partners using the newborn care training modules, and which have now been accepted as national documents intended to be rolled out to other parts of the country. The MoH and GHS should collaborate to formulate and approve a detailed implementation plan and budget for the integration of the newborn care training package into the pre-service, postgraduate and continuing education systems. The in-service training should also be reinforced for those already in the field. Sub-national (region and district) level: 6. The District Directors should collaborate with the District Assemblies (DA) to ensure that newborn care issues become a standard agenda on district quarterly review meetings. This implies advocacy work using neonatal mortality data from the district statistics so as to inform the DA on the urgency of mobilizing funds to address newborn care issues as a priority in the district. The DA should have a local budget line for newborn care as a sustainable financing solution for both maternal and newborn care services within the district. This will help the district address a number of challenges related to newborn care activities such as incentive and motivation for volunteers and CHO/CHN, fuel supply and maintenance for motorbikes used for home visits, and the supply of bicycles for volunteers. The financial contribution of the district to newborn care activities will encourage MoH/GHS to be supplemented with the recruitment of additional human resources which are currently in short supply (nurses, midwives and pediatrician) and provide the needed equipment such as new motorbikes for facilities and materials for NCU. 10

11 7. Regions and districts could set up a community performance-based financing with performance reward approach where community volunteers, providers and sub-district teams will be given money for anticipated performance. An agreement will need to be established with each district and group of providers to implement newborn care activities with well-established results and coverage for which funds and financial incentives will be provided based on performance in terms of percentage of expected results achieved. This will be an option for better coverage and achievements with newborn care. 8. Institutionalization of perinatal death audit and newborn death audit would be fundamental to ensure increased attention to newborn care and the causes and circumstances of neonatal death in order to address them more effectively and reduce neonatal mortality. It will also help avoid neonatal deaths due to poor performance or mistakes or inappropriate action of the providers. To prevent those unnecessary neonatal deaths, Regional Health Management Teams, Hospitals and the MoH/GHS should integrate newborn care indicators to the existing M&E system to monitor performance, progress, facility neonatal deaths and achievements in newborn care by providers and volunteers. This could be reinforced with provision of newborn care registers for hospital, facilities and community visits. Regions and districts health managers should be encouraged to use effectively these newborn indicators and newborn death audit results in planning and implementation of health service decisions as well as in assessing staff performance. 9. Regional and district leadership should be strengthened to drive the newborn agenda and provide support for its implementation. The district directors should be tasked to develop a comprehensive plan with costing for capacity building and refresher training schedules for staff involved in newborn care. They must maintain a register of staff and track staff movement in order to manage any capacity gap arising so as to reduce staff attrition. Further steps should be taken to provide the necessary conditions to retain trained service providers in the deprived communities. Part of the available resources should be used for incentive and motivation of volunteers and CHO/CHN to do more home visits. Regional and district leadership should reinforce the home-based postnatal care as part of routine activities and demand accountability from the CHOs/CHNs by periodically assessing their home visit registers. The number of newborns visited at home at day 3 and day 7 within 0-7 days following birth should be included in the performance appraisal of the CHOs and CHNs. Cross Sectoral Support: 10. Quality Assurance (QA), Quality Improvement (QI) and access to quality newborn care services are important instruments that need to be deployed to attain the MDGs. MoH/GHS is already putting in place a QA/QI system for health service delivery. This should be extended to newborn care services at all levels, including at the facility levels with providers and at the community level with volunteers. There should be an external and internal newborn care quality audit system. The proposed system will increase competition and motivate staff to better perform. It should also provide the needed supportive supervision and mentorship to improve the quality of the newborn care services offered by the health providers. 11

12 11. National and subnational levels should support effective documentation, communication and advocacy activities for newborn care. They should develop a communication strategy for newborn care with a costed implementation plan. Further steps should be taken to document newborn stories in the field (what is happening? what is new?) then record them and use media to disseminate them. With support from local community members they should document bad perceptions towards newborns especially neonates, negative sociocultural practices and address them with behavioral communication, education of mothers, husbands, in-laws and families during home visits and social/community mobilization. They should also intensify health education involving community members, opinion leaders, traditional and religious leaders to recognize the importance of care requirements for newborns and mothers in order to improve survival rates. They should consolidate the gains in C4D activities on newborn care using community volunteers in order the generate demand. This will require the involvement of the District Assemblies to motivate the volunteers actively involved in C4D to ensure that they continue the homebased newborn care activities. 12. National and subnational levels should support secondary data analysis to identify barriers to newborn care and address them and use operational research results to put more evidence on the table in order to support the mobilization of funds and advocacy for newborn care. Steps should be taken to address the issue of gender and ensure greater male involvement. A human rights-based approach and equity should be part of the sub-national implementation of newborn care activities. A system that will contribute to the sustainability of newborn care activities and use quality improvement method at facility level with rewards to regions, districts and selected providers that are improving newborn care should be implemented. 12

13 1. Introduction 1.1 Background The days and weeks following childbirth the postnatal period is a critical phase in the lives of mothers and newborn babies. Major changes occur during this period, which determine the wellbeing of mothers and newborns. Yet, this is the most neglected time with regard to the provision of quality services. Lack of appropriate care during this period could result in significant ill health and even death. Most infant deaths occur during this time. The number of child deaths worldwide has declined markedly in recent decades, largely through interventions to lower mortality after the first month of life. The mortality rate among children under five years of age has fallen globally by 47% (from 90 deaths per 1000 live births in 1990 to 48 deaths per 1000 live births in 2012), but the neonatal mortality rate (NMR) decreased only by 37% (from 33 deaths per live births to 21 deaths per 1000 live births) over the same period and represented, in 2012, 44% of the total under five mortality [1]. The global annual average rate of reduction in NMR since 1990 has been 2.0%, lower than that of maternal mortality (2.6%) and under-five-year old mortality (2.9%) [2]. In Ghana, around 38 per cent of under-five deaths and 60 per cent of infant deaths occur during the newborn period. According to the 2011 Multiple Indicator Cluster Survey, the U5MR was estimated at 82 deaths per 1000 live births that means 82,000 children die before reaching 5 years. Out of these, 32, 000 die in the newborn period resulting in a neonatal mortality rate of 32 neonatal deaths per 1000 live births [3]. An Emergency Obstetric and Neonatal Care (EmONC) assessment conducted by the Ghana Health Service (GHS) in 2010 reported birth asphyxia as the major cause of intra-partum neonatal death (41 per cent) at the facility level [4]. This is different than global causes where prematurity is a major cause. In Ghana, however, birth asphyxia as major cause suggests that there is an issue of quality of care at the facility level. It is thus critical to respond to these major causes of neonatal deaths in order to accelerate neonatal mortality reduction. The global response to end preventable newborn deaths led to the launch of a Global Newborn Action Plan in June 2014 [5]. Targets have been set to reduce mortality rates and WHO and UNICEF lead this work. This plan is bold and calls for a global Neonatal Mortality rate of 7 per 1000 live births by In Ghana, a number of policy responses were initiated including development of a National Child Health Policy ( ), which provides the framework to improve child survival along the continuum of care for mother and child and MDG5 Acceleration Framework and Action Plan (MAF) 2011, which identified and prioritized three key areas of intervention: family planning, skilled delivery, and EmONC for saving the lives of mothers and babies [6]. The Health Sector Medium- Term Development Plan, has also outlined improvement of access and quality of maternal and newborn care as one of the critical interventions. In line with the National Child Health Policy, UNICEF with funding support from the Government of Japan has been providing technical and financial assistance to Ghana Health Service at the National level and in a select fourteen districts of Northern and Upper East regions since October 2011 to implement the project entitled Accelerating efforts to reduce maternal, neonatal and child mortality in the Northern and Upper East regions of Ghana. The project was implemented in 13

14 two phases. Phase 1 covered the period September 2011 to December 2013 in 11 districts in the Northern and Upper East regions whereas phase 2 started in January 2013 and ended in December 2014 in the same focus districts as phase 1, but with the addition of three new districts in the Upper East region resulting in a total of fourteen project districts. The key components of the project included: Minimum of two home visits (to mother and newborn) within the first 7 days after delivery by appropriately trained community health workers; Integrated Management of Neonatal and Childhood Illnesses (IMNCI) at facility and community levels; Scale up of Basic Emergency Obstetric and Newborn Care (BEmONC) at Community Health Planning and Services (CHPS) and Health Centres; Developing capacities on life-saving skills for midwives; Promotion of key household and community practices related to delivery and newborn care; Leveraging existing resources and initiatives in the project area; Using a systems strengthening approach to enable sustainability beyond project period To provide evidence on the effectiveness of this project for possible scale-up, an independent formative evaluation was commissioned with funding from UNICEF Ghana. The aim of this formative evaluation is to assess key components of the project. 1.2 Specific objectives of the formative evaluation: 1. To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child survival interventions with a focus on community-based care (Home-based Postnatal Care (HBPNC)) and facility-based integrated Management of Newborn and Childhood Illnesses approaches in the selected districts of the Northern and Upper East regions; 2. To ascertain the project s contribution for capacity building, developing training resources and facility structures to respond to the high levels of newborn morbidity and mortality in selected districts of the Northern and Upper East regions; 3. To ascertain the effectiveness of the project s evidence-based advocacy for issues related to newborn survival and national policy environment, 4. To draw lessons on the implementation capacity for national scale-up of the essential newborn care model through home-based early postnatal care. 14

15 2. Scope of the Evaluation and Evaluation Questions It is expected that the project partners Ministry of Health (MoH), GHS, UNICEF and JICA will use the findings of the evaluation in their different capacities and functions, to develop future plans and interventions and to inform policies and strategies. 2.1 Scope of the formative evaluation The scope of the formative evaluation covered the areas of implementation of the project in the two regions (Northern and Upper East Region) of Ghana. This evaluation also expanded its scope to the national level, to ascertain its sphere of influence on the overall maternal newborn and child health (MNCH) programming in Ghana. The evaluation covered the overall Government of Japan, UNICEF and Government of Ghana s partnership on the newborn health programming from the two phases of the project from September 2011 to December The evaluation focused on, and included the following beneficiaries and stakeholders in the process: Final beneficiaries: newborn babies, mothers and caregivers; Service providers: healthcare professionals whose capacity had been built (including doctors, midwives, community health nurses and sub-district health professionals); Sub-national decision-making level: District and Regional health authorities; National decision-making level: national authorities and key stakeholders (MoH, GHS, Development Partners JICA, USAID, EC, PATH, JHPIEGO, UN System- UNICEF, WHO, UNFPA, National Newborn Care committee); National Professional Societies and Academia: Paediatric Society of Ghana, Society of Obstetricians and Gynaecologists of Ghana, Ghana Registered Midwife Association, Teaching Hospitals, School of Public Health. 2.2 Evaluation questions: The formative evaluation attempted to provide answers to a number of questions to meet Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC) evaluation criteria of relevance, effectiveness, efficiency, and sustainability including UNICEF s Coherence and Human Rights-Based Approach (HRBA) to Programming and Equity for all target groups. Following an inception meeting with the National Newborn Sub-Committee and other stakeholders at the national level, the evaluation questions were reviewed using the Terms of Reference as the basis (See Table 2.1). 15

16 Table 2.1: Evaluation criteria, targets and questions Criteria Relevance Effectiveness: Questions/Target groups National decision-making level: Is the intervention relevant in terms of alignment of project objectives with national strategy and stakeholder s priority and needs? National decision-making level: To what extent has the project contributed to the policy direction of the National Newborn Strategy and Action Plan in terms of advocating for and facilitating to bring newborn health onto the national agenda of MNCH Programming? Sub-national decision-making level: To what extent has the project contributed to strengthen capacity of regional health management teams and district health management teams for planning, informed decision making and prioritization of the newborn health as per the National Child Health Policy ( ) and other national guidelines and protocols? Service providers level: To what extent are the established level-two (without ventilator and incubators) newborn care units in six District Hospitals perceived to have improved the management and survival of sick newborn babies? Which are the enabling/constraining factors that facilitated/hindered the management of sick newborn babies in District Hospitals? To what extent has there been an improvement in quality of care during post-natal care in the health facilities targeted by the project? To what extent has the training and mentorship component of the project responded to capacity building needs of the different levels of service providers? Final beneficiaries level: To what extent do beneficiaries report to have been reached by project communication and social mobilization interventions, like community durbars, mother support groups, community-based agents (CBA) and Red Cross mothers (in the Upper East Region)? To what extent do beneficiaries report an improvement in their newborn care and health seeking practices (ANC, PNC, well baby clinic) as a consequence of improved counseling by Community Health Officer (CHO)/Community Health Nurse (CHN), CBA, Red Cross mothers and mother support groups? To what extent has the intervention contributed to improve health and wellbeing of newborn babies? Efficiency: Sustainability: National level: National level: Were the allocated resources used efficiently to achieve the project objectives? Are the available resources adequate to meet project needs? Have policy makers at MoH/GHS demonstrated ownership over the different interventions related to newborn survival? Has the Government of Ghana prioritized the health and wellbeing of newborn babies in the government s policy documents (Ghana Shared Growth and 16

17 Development Agenda for , Medium Term Health Expenditure Plan for ) and allocation of resources (budget line on approved Ministry of Health budget) for newborn health? Sub-national decision-making level: Have District Health Management Team (DHMT) and District Assembly (DA) demonstrated ownership and capacity for resource mobilization to be able to selfsupport and consolidate the achievements and the expansion of newborn health interventions? Service providers level: Can the commitment and motivation of CHO/CHN and community volunteers that were enhanced through the project last for a continued provision of home-based services to mothers and their newborn babies? What are the bottlenecks and barriers for home-based postnatal care within the framework of continuum of MNCH care? Coherence: Human rightsbased approach (HRBA): Criteria Relevance Final beneficiaries level: National National Can the behavioral changes among beneficiaries on essential newborn care be sustained? Has the project facilitated synergies and avoided duplications with interventions and strategies promoted by other UN agencies and development partners (JICA, USAID, EC, PATH and others) within the National Child Health Policy and MAF? Has the project given due importance to donor s (Government of Japan) visibility in line with UNICEF s donor visibility guidelines; Has the project incorporated the HRBA to programming? Has the project considered the equity approach (i.e. focus on most deprived areas, areas with high prevalence of critical newborn and under-five mortality, low income families) as well as facilitated the reduction of access barriers to MNCH services by the target group? Questions/Target groups National decision-making level: Is the intervention relevant in terms of alignment of project objectives with national strategy and stakeholder s priority and needs? 17

18 Effectiveness: National decision-making level: To what extent has the project contributed to the policy direction of the National Newborn Strategy and Action Plan in terms of advocating for and facilitating to bring newborn health onto the national agenda of MNCH Programming? Sub-national decision-making level: To what extent has the project contributed to strengthen capacity of regional health management teams and district health management teams for planning, informed decision making and prioritization of the newborn health as per the National Child Health Policy ( ) and other national guidelines and protocols? Service providers level: To what extent are the established level-two (without ventilator and incubators) newborn care units in six District Hospitals perceived to have improved the management and survival of sick newborn babies? Which are the enabling/constraining factors that facilitated/hindered the management of sick newborn babies in District Hospitals? To what extent has there been an improvement in quality of care during post-natal care in the health facilities targeted by the project? To what extent has the training and mentorship component of the project responded to capacity building needs of the different levels of service providers? Final beneficiaries level: To what extent do beneficiaries report to have been reached by project communication and social mobilization interventions, like community durbars, mother support groups, community-based agents (CBA) and Red Cross mothers (in Upper East Region)? To what extent do beneficiaries report an improvement in their newborn care and health seeking practices (ANC, PNC, well baby clinic) as a consequence of improved counseling by Community Health Officer (CHO)/Community Health Nurse (CHN), CBA, Red Cross mothers and mother support groups? To what extent has the intervention contributed to improve health and wellbeing of newborn babies? Efficiency: Sustainability: National level: National level: Were the allocated resources used efficiently to achieve the project objectives? Are the available resources adequate to meet project needs? Have policy makers at MoH/GHS demonstrated ownership over the different interventions related to newborn survival? Has the Government of Ghana prioritized the health and wellbeing of newborn babies in the government s policy documents (Ghana Shared Growth and Development Agenda for , Medium Term Health Expenditure Plan for ) and allocation of resources (budget line on approved Ministry of Health budget) for newborn health? Sub-national decision-making level: Have District Health Management Team (DHMT) and District Assembly (DA) demonstrated ownership and capacity for resource mobilization to be able to selfsupport and consolidate the achievements and the expansion of newborn health interventions? 18

19 Service providers level: Can the commitment and motivation of CHO/CHN and community volunteers that were enhanced through the project last for a continued provision of home-based services to mothers and their newborn babies? What are the bottlenecks and barriers for home-based postnatal care within the framework of continuum of MNCH care? Coherence: Human rightsbased approach (HRBA): Final beneficiaries level: National National Can the behavioral changes among beneficiaries on essential newborn care be sustained? Has the project facilitated synergies and avoided duplications with interventions and strategies promoted by other UN agencies and development partners (JICA, USAID, EC, PATH and others) within the National Child Health Policy and MAF? Has the project given due importance to donor s (Government of Japan) visibility in line with UNICEF s donor visibility guidelines; Has the project incorporated the HRBA to programming? Has the project considered the equity approach (i.e. focus on most deprived areas, areas with high prevalence of critical newborn and under-5 mortality, low income families) as well as facilitated the reduction of access barriers to MNCH services by the target group? 19

20 3. Evaluation Methods 3.1 Ethical considerations Individual consent was obtained before interviews or focus group discussion (FGD) were conducted. Each participant was told the purpose and use of the information being collected by the evaluator. Questions were posed to collect their opinions and views about the activities of the newborn projects and their possible impact on newborn health indicators. Before the start of the interview or FGD, participants were told that their participation was voluntary and that they were free to withdraw from the interview or FGD at any time. They were also told that the information provided would remain confidential and used anonymously. Participant who gave their consent were given an informed consent form for signature (See Appendix AP1). Interviews and FGD were conducted in a private and isolated place to ensure confidentiality and provide a comfortable environment. Evaluators did not seek personal information, or opinions believed to be controversial. No risk is expected for participants since the main aim of the evaluation is to improve newborn health (See Appendix AP2a0). 3.2 Study design The formative evaluation employed a mixed method design consisting of qualitative and quantitative components. The qualitative component consisted of in-depth interviews with key national decisionmakers, donor partners, sub-national authorities and service providers, and focus group discussions (FGD) with mothers or caregivers drawn from the communities in one project district. Interview guides (See Appendices AP2a1, AP2b1, AP2c1) and the FGD guide (See Appendix AP2d) were developed for the in-depth interviews and FGDs respectively, based on the evaluation questions along the OECD/DAC evaluation criteria for relevance, effectiveness, efficiency, and sustainability. It also considered the two additional criteria of interest to UNICEF namely: Coherence and Human Right-Based Approach to Programming and Equity. The interview guides were translated into semi-structured questionnaires (See Appendices AP2a2, AP2b2, AP2c2) with Likert scale responses (i.e. 1=fully disagree; 2=disagree; 3=no opinion; 4=agree; 5=fully agree) to quantify the degree of agreement, or disagreement to a set of statements or declarations drawn from the interview guides. Respondents, were however, not prompted for the reasons of their disagreement or their lack of opinion. The questionnaire was administered to as many stakeholders and service providers as possible including those who took part in the in-depth interviews. These questions were first pre-tested for their validity and their ability to elicit the right kind of responses, they were then reviewed (as necessary) prior to being finalized for the main fieldwork. The quantitative component involved the abstraction of neonatal health indicators from the District Health Information Management System (DHIMS (2) and other relevant data sources based on indicators developed from the evaluation questions as well as project-specifics to assess the project s success and effectiveness. Service output indicators were extracted for the region as well as the project districts. Due to the rarity of certain events, impact indicators such as neonatal deaths per 1,000 live births were extracted as a regional level indicator for analysis. 20

21 3.3 Sample size consideration and sampling For the quantitative component, all 14 project districts were included in the extraction of key neonatal health indicators. For the qualitative component, in-depth information were collected from key target groups. Since this was a qualitative research with recorded interviews, the evaluation team planned to carry total number of interviews, for the entire evaluation exercise, of less than 45 for the two regions including FGD sessions and at the national level but with adequate representation of all levels of the target groups. At the national level, 12 key informants (See Appendix AP4) were contacted for in-depth interviews in consultation with UNICEF Team. For the two regions, sampling for the in-depth interviews and FGD sessions happened in two steps. First, the 7 districts in each project region were stratified according to agreed criteria with stakeholders (Urban with hospital where there is a Neonatal Care Unit, Rural with hospital, and Rural without hospital). One district was randomly selected from each of this stratum giving a total of 3 districts for the interviews or FGD. If a stratum had only one district, that district was included by default. The results of the selection were Kpandai, Savelugu-Nanton and Tolon-Kumbungu in the Northern region, and Bolga Municipal, Kasena-Nankana West and Talensi- Nabdam in the Upper East region. However, after a meeting between the evaluation team and the regional director in the Northern Region, it was agreed to replace Kpandai with Bole since Bole and Kpandai shared similarities in their performance for newborn care, but Bole had more frequent and active home-based visits and social mobilizations compared to other selected sites. Having Bole would cover that aspect of home visits as a key component of newborn care. Talensi-Nabdam was also replaced with Bawku West in consultation with the regional director of the Upper East Region, because the evaluation team thought that it was better to have representation of the districts that experienced the two phases of the project. Afterwards, 2 facilities (1 Health centre and 1 CHPS compound) were selected in consultation with the District Director based on the fact that the facility in-charge (Midwife or CHO) had been in post for the duration of the project. The midwife, CHN or CHO in each selected facility was contacted for an in-depth interview. For the final beneficiaries in Bolga and in Savelugu districts, the evaluation team, in consultation with the regional director, agreed to draw 8 to 12 mothers of newborn babies from the communities with the assistance of the district director and community health officers. The CHO explained the aim of the FGD to them, and upon obtaining their consent, they were asked to report to the sub-district health centre. At the health centre, the evaluation team sought their consent to participate in a FGD session. Drawing the mothers from the project communities was preferred to the proposed approach of selecting PNC registrants from a NCU facility due to the possibility of sickness of the mothers babies which may result in a state of distress, thus corroding their ability to participate in a FGD. The regional directors in the project regions and the district directors in the selected districts were also interviewed. These amount to a total of 28 in-depth interviews and two FGDs carried out (Table 3.1). 21

22 Table 3.1: Sample size for the in-depth interviews and focus group discussions among sub-national, service providers, and final beneficiary target groups Region Selected districts Number of In-depth interviews Regional Director/ NBC Coordinator District Director/ NBC Coordinator Midwife /CHO/ CHN Number of FGDs Northern Bole Northern Savelugu-Nanton Northern Tolon-Kumbungu Upper East Bolga Municipal Upper East Kasena Nankana West Upper East Bawku West Total Total 3.4 Field work/data collection For the quantitative component of the evaluation, key neonatal health indicators at the district level for all the 14 project districts and at the regional level were extracted from DHIMS (2) over the period spanning from 2011 to 2014 and submitted to consultants through a formal request by UNICEF Ghana to the Director-General of the Ghana Health Service. Two NCU facilities (one in each project region) were visited to assess their capacity for newborn care particularly focusing on referral-in and referral-out indicators. For the qualitative data, the evaluation team had in-depth interviews with 12 key national decision makers and health partners. Following this, the evaluation team made a two-week field visit to six districts in the two project regions and had in-depth interviews with sub-national authorities and service providers as well as FGD with women beneficiaries. Table 3.2 in the appendix AP5a shows the work schedule of the consultants. 22

23 Figure 3.1: Maps of the Northern and Upper East regions with the 14 districts of newborn care intervention and 6 selected districts for interviews. DISTRICTS OF THE NORTHERN REGION Districts are colored for emphasis DISTRICTS OF THE UPPER EAST REGION Districts are colored for emphasis 23

24 3.5 Data Analysis For the quantitative component, indicators were defined for each evaluation question (where appropriate). The percentage change in key newborn health indicators over the period spanning from 2012 to 2014 was estimated to ascertain the effect of the project in improving newborn survival. Where the rate of change in the indicator in the project districts was faster than that in the entire region over the same period, it was suggestive of the intervention being effective. Also, examining coverage trends was essential for assessing project progress. Information on trends required at least two separate and comparable measurements at two points in time. A measure of progress the coverage gap defined as how much coverage would need to increase from the 2013 level to reach universal coverage was estimated to examine coverage trends. The change from 2013 to 2014 was then expressed as a percentage of this gap. For the qualitative data, recorded interviews were transcribed verbatim. Data were analyzed manually using two analytic approaches, namely: (1) Thematic analysis looking for themes and patterns among data (verification); and (2) Narrative analysis in order to identify narratives or cases, and explore how they differed between groups. No qualitative data analysis software was used for this evaluation. The ideas, views, opinions and quotations from the transcribed (verbatim) data summaries and the notes, were used to illustrate the reports by evaluation criteria. For the focus group discussion, a Matrix for assessing the level of consensus in the focus group was used (Table 3.2). Data from the semi-structured questionnaires were summarized using proportion of respondents with degree of agreement or disagreement to a set of statements or declarations. Table 3.2: Matrix for assessing level of consensus in focus group discussion Focus Group Member 1 Member 2 Member 3 Member 4 Member 5 Question The following notations were entered in the cells: A = Indicated agreement (i.e., verbal or nonverbal) D = Indicated dissent (i.e., verbal or nonverbal) SE = Provided significant statement or example suggesting agreement SD = Provided significant statement or example suggesting dissent NR = Did not indicate agreement or dissent (i.e., nonresponse) 3.6 Data archiving All data collected during this evaluation exercise including recorded interviews (in MP4 format) and transcribed data (in Word format and electronic version) were submitted to UNICEF Ghana Office for archiving. 3.7 Quality Assurance Researchers took appropriate and necessary measures to ensure the quality of the data collected from the key informant interviews by minimizing ambiguity when presenting the questions to the 24

25 interviewees. As in many health information systems, the researchers recognized the limitations of the quantitative data extracted from the DHIMS (2) and tried within the limited time for this exercise to validate suspected figures with facility records. Researchers also recognized the limitations of in-depth interviews and FGDs in terms of the small sample size or the limited number of participants interviewed. The information collected was therefore analyzed and interpreted within their contextual thematic scheme and the individual opinions and views expressed. 3.8 Challenges and limitations As mentioned in section 3.7, extracting quantitative data from the DHIMS may have posed some limitations relating to the completeness and accuracy of the data. Researchers used other sources whenever possible to validate suspected figures. Qualitative data have their own limitations in terms of sample size and generalization to the entire population under study. For this evaluation, opinions and views collected during in-depth interviews and FGDs with limited number of participants were therefore analyzed and interpreted within that context of individual ideas and appreciations. In the absence of control areas, it is difficult to attribute observed changes and achievements entirely to implementation of the newborn care project which did not cover all the districts in each region. However, examining changes from one period to another and trends over a period of time is bound to assess the contribution of the NBC project when quantitative data is available. The replacement of Kpandai district with Bole district which had very active home visits and social mobilization may be seen as bias. This choice by the evaluation team and regional authorities offers the advantage to better assess potential contribution of social mobilization to improving newborn health in addition to home based care. 25

26 4. Evaluation findings This chapter presents the results of the evaluation exercise organized by four evaluation objectives taking into account the evaluation criteria. For each of the four evaluation objectives and the relevant evaluation criterion the evaluation finding was quoted and the available evidence was provided as an explanation. The available evidence was gathered from the desk review, DHIMS (2)/facility records, semi-structured questionnaires, in-depth interviews and the focused group discussions. Objective 1: To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child survival interventions with a focus on community-based (Home-based Postnatal) Care (HBPNC) and facility-based integrated Management of Newborn and Childhood Illnesses approaches in the selected districts of the Northern and Upper East regions; The established level-two newborn care units in District Hospitals have contributed, to some extent, to an improved neonatal survival through an improved management of sick newborn babies. However, challenges such as cases from the communities arriving late for management still remain. For example, institutional neonatal deaths per 1,000 live births reduced by about 51% (6.9 to 3.4 ) in the Northern region and by about 43% (5.8 to 3.3 ) in the Upper East region over the project period spanning from 2012 to 2014 (Figure 4.4). However, it should be noted that the project was implemented in well identified districts and not in the entire region which contains several more districts. Disaggregated and analyzed by project district, these findings show mixed results and are presented in table 4.1 in the appendix AP5a. The evidence from one neonatal care unit suggests improved management and survival of sick newborns. For example, the NCU at Bolgatanga Regional Hospital in the Upper East recorded downward trends in neonatal deaths per total admission since the start of the NCU in January 2014 (Figure 4.4a). It is possible that the skills acquired during the training are being applied in managing sick babies referred to the NCU. However, there appeared to be stagnation in the Savelugu District Hospital NCU in the Northern region (Figure 4.4a). A well-conducted death audits would bring out the reasons for such a stagnation. 26

27 Figure 4.4: Percentage change in institutional neonatal deaths per 1,000 live births over the period in the project regions. There was a general view among service providers (27 out of 32) that the established level-two newborn care units in Hospitals improved the management and survival of sick newborn babies (Figure 4.4b). However, three of them disagreed and two had no opinion probably because of the challenges. 27

28 The major element of success with NCU is the supply of equipment for saving the lives of preterm babies, low birth weight babies, and babies with hypothermia or asphyxia. This equipment included the resuscitation machine, the incubator, the baby warmer and the phototherapy machine. Other important elements are the Kangaroo mother care, the body temperature controller, the Random blood sugar test, the oxygen concentrator, and the fluid therapy. The midwives in two NCUs assessed during the field visit mentioned some factors as having contributed greatly to the reduction in neonatal deaths in the facilities. However, the statistics from the facilities were not available to the evaluators to illustrate the impact of the equipment in the NCU on the survival of premature newborns as stated by the midwives. In spite of these achievements there remains serious challenges at the NCU in terms of staff attrition a number of staff trained in newborn care have left following their posting elsewhere or to further their studies. For example, in the Upper East region out of 6 staff trained who started the NCU, only 3 were still working in the NCU during this evaluation. The other challenge is the lack of oxygen cylinders. UNICEF brought more oxygen concentrators to the NCU. Participants also mentioned the breakdown of the incubators, which they have no local capacity to repair. The administration was, however, in contact with Accra which deployed a technician to repair the malfunctioning device. Strategies to educate mothers to leave their babies in the NCU is also a challenge. This is because mothers do not like to be separated from their babies at the hospital. This highlights the need to have room in the facilities for mothers to stay. Sometimes, staff run out of key medicines for the newborn and have to prescribe them. The Northern Region also faces some challenges as stated in the quote below: Another challenge is the space available for NCU which is too small. Normally when babies are referred to us from home, we are not supposed to put them together with those babies born in the hospital, but with limited space we have no choice than to put them together. We also need a pediatrician for our NCU so that the cases we now refer to Tamale Teaching Hospital can stay with us. Most of our client mothers do not like to be referred. (Provider of NCU in the Northern region) The beneficiaries expressed satisfaction with the home-based care their babies received from the CHOs/CHNs. They also indicated that they received basic counseling on health and wellbeing of mothers and babies and that the regular interaction with the CHOs/CHNs and community volunteers provided the opportunity to share their experience and bring up their challenges on providing care to their newborn babies, and taking care of their own health. The results suggest that through sustained counseling and communication for development activities, mothers were informed on the importance of kangaroo mother care to ensure the survival of low birth weight babies. 28

29 FGD mothers in the Northern region told us that they now report diarrhea cases, vomiting and fever to the health facility and found that the treatments at the facility were effective to stop sickness in the child. They emphasized that when they applied what they have learnt, they found that babies were no longer falling sick and both mother and baby were in good health. This is in line with improvements in newborn care and health seeking practices reported by the beneficiary mothers. The quote below from a mother in the Upper East is reinforcing the observed improvements. From the pictures we learnt that after using toilet we should wash our hands before touching the baby and also before breastfeeding the baby. Putting all these into practice helped us to keep the child away from sickness. Before many pregnant women were delivering at home. These days more women are delivering in health facilities and we no more see mothers and babies dying when we give birth there. (FGD participant, Upper East region) Table 4.3:Trends in uptake of kangaroo mother care, Half Year (HY) 2011 to 2014, Upper East region Indicators 2011 HY 2012 HY 2013 HY 2014 HY % change (2014 vs 2013) No. LBW Babies ,138 1,297 % LBW No. on KMC % of LBW on KMC Source: regional statistics data 29

30 Photo 4: A Focus Group Discussion session in Upper East Region, 17 th January 2015 Perceptions and reported statements from beneficiaries are that the health and wellbeing of babies have improved substantially to their satisfaction. They reported positive behavioral changes and less diseases due to the newborn care intervention especially education by the nurses and volunteers during home visits and supportive communication activities. CHO and CHNs focused on improving the quality of newborn care provided during the postnatal home visit. The long-term expectation is that every newborn will receive home-based care by a trained health worker. In the Northern region, the percentage of babies visited at home by trained health workers increased from about 30% in 2013 to 37% in 2014, representing a 10.5% gap closed to achieve universal coverage of all newborn with home-based care (Table 4.4). The Saboba district was the highest performer (53.8% of gap closed) with Yendi being the worst performer, which recorded a reduction in home-based visits. In the Upper East region, the percentage of babies visited at home by trained health workers increased from about 39% in 2013 to 58% in 2014, representing a 30% of gap closed to achieve universal coverage of all newborn with home-based care (Table 4.4). Bawku West was the highest performer with 45% of gap closed. 30

31 The strategy of using community volunteers, who are very familiar with the communities and persons within the communities, as agents to identify households where pregnant women are, and where recent deliveries have occurred and inform and link-up with the CHOs/CHNs for scheduled visits was an enabling factor for the home-based postnatal newborn care. Table 4.4: Percentage of newborn babies who were visited at home by trained health worker Region Project Districts Northern Expected target neonates (i.e. 80% of expected deliveries) Number of babies visited by health worker % of babies visited 2013 (A) 2014 (B) % of gap closed =[B- A]*100/[100- A] Bole Gushiegu Kpandai Saboba Savelugu-Nanton Tolon-Kumbungu Yendi Total All seven districts Upper East Bawku Municipal Bolga Municipal Kasena Nankana East Bawku West Garu Tempane Kasena Nankana West Nabdam Talensi Total All seven districts Source: Home-based Postnatal Newborn Care Report, Started reporting January 2014 Focus group discussions with beneficiaries who were mothers selected from rural communities of the Northern and Upper East regions reported significant positive behavioral change towards newborn care among the population as a consequence of home-visits and education by nurses and volunteers, and supportive communication activities of the intervention. Most of them reported that the days of children with many diseases are now behind them. They expressed hope in the future of their babies and wished that the home visits would continue. The following are illustrative statements by mothers during the FGDs: 31

32 There has been major improvement in the lives of our babies. Our children are no longer falling sick as it used to be. They are no longer dying or getting sick or having skin diseases or eye diseases. We now go to the clinic to give birth. (FGD participant Upper East region) Participants in the Northern Region also reported that the health and wellbeing of their babies had improved and their children were no longer as sick as they were in the past, thus affording them time to go to the market to sell their wares and do business, or perform farming activities to support their families instead of worrying, or having sleepless nights when their babies are sick. This expressed happiness was shared by many of the mothers during the FGD sessions. One mother reported that when she gave birth to a low birth weight baby, the nurse visited her at home and referred her to the NCU where she was taught Kangaroo mother care, exclusive breastfeeding and other general newborn care practices. Today the baby has put on weight and is doing well. The quote below reinforces the perceived improvements in the health and wellbeing of the babies. There have been changes. Hygiene practice is really helping since our children are no longer having diarrhea and cholera as in the past. We were taught first aid for fever in child. Putting this into practice has reduced convulsions in children when they have fever. Now it is far better than what it used to be. Children are no longer dying as we used to see. Immunization is also helping to have fewer deaths. (FGD participant Upper East region) Beneficiaries also reported significant improvement in mothers health due to the intervention, which sensitized and mobilized pregnant women to go to health facilities for antenatal care and facility deliveries. FGD participants noted that the newborn care project is of great psychological helps to the mothers. Since it reduces the frequency and severity of sickness in babies and children, it alleviates the psychological and physical burden on mothers who no longer have to worry or spend sleepless nights with their sick children as was the case in the past. These days, the babies are healthier and mothers can stop worrying and carry out their daily economic activities. As reported by a mother during the FGD: It is better now than before when many pregnant women were delivering at home. These days more women are delivering in health facilities and we no more see mothers dying when giving birth, even breach delivery is taking place in facility without complication and both mother and babies are being saved. (FGD participant Upper East region) 32

33 The project considered the equity approach to programming to some extent because project interventions were targeted at poor and disadvantaged regions with relatively high maternal and newborn mortality. A review of the project proposal indicated that due considerations were given to changes in the burden of neonatal deaths during the selection of the project regions. It also indicated that the regions were chosen to consolidate the gains recorded following a series of interventions in the past However, if the newborn care project in the Northern and Upper East regions has to ensure full equity, it would necessitate heavy infrastructure, human resource distribution, transport and equipment investments. For optimal results in a resource-constrained country, it is critical to prioritize activities and focus on the worse performing areas in order to accelerate the bridging of the inequality gap. A national key informant attested the following on equity considerations in programming. Sometimes we need to concentrate our efforts into just very few places. For example, in just Greater Accra or Central region where facilities are better, but need equitable distribution of equipment and human resources. Then we look at the data that we have, if many neonatal deaths are occurring in the Central region then it becomes our priority. This means that we target problem areas where the burden is, concentrate our efforts in those areas where help is needed and build up capacity without neglecting the other areas. (National key informant) The national newborn strategy provides the framework for priority newborn activities, which can be operationalized through annual plans and budgets with the support of health partners. A review of the Medium Term Health Expenditure Plan for and the Ghana Shared Growth and Development Agenda for showed that many activities can directly and indirectly impact the lives of the newborns. The National Newborn Strategy provides a more focused framework on newborn survival, which can be operationalized through annual plans and budgets with the support of health partners. Eleven (11) out of 14 national decision-makers were of the opinion that newborn issues have been given priority attention in the national and sector policy documents (Figure 4.10) but three (3) of them gave no opinion or disagreed. 33

34 Objective 2: To ascertain the project s contribution to the capacity building, developing training resources and facility structures to respond to the high levels of newborn morbidity and mortality in selected districts of the Northern and Upper East regions; The project supported capacity building workshops for all district directors of health services and public health nurses in the project districts on essential newborn care. They are now able to plan for newborn care activities. A review of the districts annual plans showed that they have all featured newborn care activities. Key informants reported that the project trained subnational personnel as trainers who in turn trained sub-district and community health service providers on newborn care. The project succeeded in putting government in the driving seat for newborn care and contributed to build the capacity of providers to deliver quality service. All the district directors of health services and district public health nurses in the project districts reported to have been introduced to, or fully trained in newborn care. 26 out of 27 sub-national decision-makers agreed or fully agreed that the project actually enhanced their capacity to plan and prioritize newborn health services (Figure 4.3). One of them, however, had no opinion about this statement. 34

35 The project succeeded in creating a national and sub-national training capacity, which could be used for in-school basic training of nurses and midwives. The development of the community-training module for community-based agents, volunteers and nurses was fast-tracked as a result of the project being implemented in the Northern and Upper East regions as mentioned by one of the national key informant below: Coming back to development of tools, we had started long ago working on the community module to train community-based workers, volunteers as well as community health officers but it was left there in a draft form. When the NBC project started the demand was high so that we had to go back and pick that draft document and get funding to refine the draft and finalize it for use in training community workers and volunteers of the project areas. Now it is printed and in use elsewhere and will be used nationwide to implement the newborn strategy. (National key informant) 35

36 Photo 2: Some training materials on essential newborn care Key informants in the Upper East who underwent the training strongly stated that it had built their capacity for service provision. They also indicated that the bottleneck analysis tool helped identify the root causes of neonatal deaths as a first step in the planning process. They reported that in the past when they did not meet their targets, they just assumed that it was due to in- or out-migration or famine, and never questioned these assumptions or attempted to understand the root causes. Another key informant in the Northern region reported that with the bottleneck analysis tool, the bottlenecks and root causes were identified and the team developed a plan with a corresponding budget to address the identified bottlenecks in newborn care. The plan was regional and every district had its micro-plan. This plan or micro-plan are also tools to engage other funders in order to help the district or region to address the bottlenecks. 36

37 The program has assisted us very well. We benefited from a number of trainings from the management level to lower level. Through the training program, we were able to use the bottleneck analysis to identify our challenges, their root causes and find way-out for controlling the challenges, we did not say we have a problem, but we have a challenge meaning that we can find ways to solve them through the system and do better. (Key informant in the Northern region) The project supported capacity building workshops for frontline health workers on essential newborn care. A total of 4,151 health workers plus community volunteers were trained on essential newborn care. However, while service providers reported satisfaction with the training content, the capacity created might diminish over time especially under high attrition situations. The capacity of a total of 4,151 health workers including Doctors, Midwives, CHOs/CHNs as well as volunteers and mother support group members was enhanced through training in life-saving skills and essential newborn care (Table 4.2). They were imparted with the requisite knowledge and skills to provide counseling, preventive and curative interventions, including referral to higher levels of care towards accelerated reduction of neonatal deaths. Table 4.2: Participants in facility and home-based postnatal care trainings during project implementation from September 2011 to December 2014 (Phase 1 and Phase 2) Types of training Project phases Facility based newborn care Home-based postnatal care Northern Region Upper East Region Phase 1 Phase 2 Phase 1 Phase 2 Total Type of participants Doctors, midwives, medical assistants, nurses CHOs and CHNs plus enrolled nurses ,748 Community volunteers 0 0 1, ,200 Red Cross motherto-mother support groups Red Cross supervisors, mother to mother support group facilitators Total 1, , ,151 Source: UNICEF Ghana-Government of Japan newborn project final report, 2013 and March out of 35 (77%) service providers whose views were sought during the field visit to the project regions agreed or fully agreed that the training and mentorship component of the project responded 37

38 to their capacity building needs (Figure 4.6). Six of them gave no opinion because they were not selected for the formal training workshop. The two who disagreed, wanted more topics to be covered by the training session. The trainees are now ready with the requisite knowledge and skills to run newborn care units at hospitals and provide home-based services to mothers and their newborns. The providers were very happy to have undergone these trainings in newborn care. The statements below from providers are an illustration. The training in newborn care was useful and helpful for my work. I go back to the guidelines and protocols to manage each case and problem very well and every day. Before the training, I knew little about newborn care and managing their sickness was difficult for me and I use to refer them. The training has added values to my skills and knowledge and now I am more confident to handle any case or complication. (Provider in the Upper East region) As a midwife I did not know much about newborn care and the training has equipped me and given me confidence to do my work in newborn care. Now I have more experience based on the guidelines and the counseling card. I feel confident to educate mothers even though I do not understand very well the local language. I use pictures and they understand. I have trained all my staff and they know how to manage newborn care and do it even if I am not there. (Provider in the Northern region) Some providers in health centers and CHPS compounds who were interviewed indicated that there were many things which they were not familiar with but have since been educated on during the trainings on newborn care. These included how to: 38

39 conduct home-based follow up or monitoring visits; talk to mothers during home visits and sometimes husband, mother- and father-in-laws; take care of the baby or examine or assess the baby and appreciate individual and environmental hygiene; keep the baby warm after delivery (skin to skin approach, kangaroo mother care etc.); position the baby at breast; care for the newborn and identify danger signs; perform resuscitation; and take care of the cord.. It would have been good to have the cost analysis of the investment in capacity building but a robust cost-efficiency analysis is beyond the scope of this evaluation, so researchers were unable to determine if the allocated resources were used efficiently to achieve the project objectives. A total of 4,151 health professionals and community-based agents were trained at the cost of eight hundred and eight thousand, three hundred fifty six (USD 808,356) US Dollars (Table 4.5) meaning that on average, the cost of training a health professional in newborn care was approximately USD Out of USD 1,812,187, 45% (USD 808,356) was spent on training and capacity building and 35% (USD 617,271) was spent on community and facility service delivery and demand generation. The indirect cost of the project is USD 109,055 (6%). Table 4.5: Financial Resource utilisation Item description Phase 1 Phase 2 1. Enhanced facility and community capacity including development of resource materials 346, , Community and facility service delivery and demand generation 394, , Strengthened monitoring and evaluation 4. Technical assistance 5. Communications and visibility 6.Cross sectoral project support 45, , , , , , , Amount in USD 808, , , , , , Total expenditure for Programme 831, , ,812, Total programmable amount 831, , ,813, Programmable balance Source: Programme financial utilization report phase 1 and phase 2 received from UNICEF

40 There has been a reported improvement in quality of post-natal care by providers Thirty-two (32) out of 35 (91%) service providers in selected facilities in the project districts agreed or fully agreed that there has been an improvement in quality of care during post-natal period in the health facilities targeted by the project (Figure 4.5). One provider disagreed with this statement and two of them gave no opinion. In terms of postnatal newborn care at the facility, the application of skills and knowledge acquired during the newborn care trainings have helped improve the quality of postnatal care. Also, supervision, monitoring and mentorship have helped improve the quality of service delivery by the trained providers as it pertains to newborn care. Following are some quotes from service providers in support of such quality improvement: There has been an improvement in quality of newborn care at the facility. Today due to the training, we no longer fear or panic when there is a case of sick newborn. We use the guidelines to manage the case. Our community encourages facility delivery because of the quality of services provided. We have recorded less neonatal deaths for the past 12 months. (Service provider in the Upper East region) Our facility report shows reduction in newborn deaths. Our admission report shows a big increase in facility deliveries and more surviving babies than before. When we identify any problem in the newborn that endanger the life of the baby we manage the case using guidelines or we refer it to NCU. Newborn survival has improved a lot. (Service provider in the Northern region) 40

41 These statements from providers need to be taken as opinions and views to be crosschecked or confirmed with existing statistics. In fact, to confirm their statements, providers indicated that prior to this project, mothers hardly came back to the facility with the baby during the postnatal period. It is only since this project started, that mothers are using more and more facilities for postnatal care. The provider said, We have referred few newborns to NCU and all of them came back well and healthy. One provider from Upper East gave her own example of improvement in quality of delivery saying: After the training, I received a case of twin delivery and the mother was bleeding. Since I was taught how to stop bleeding, I gave her oxytocin I.V and was able to stop the bleeding. If it was before the training, I would have just referred the woman to the district hospital. (Service provider in the Upper East) Another provider from the Northern region gave her example of a low birth weight baby weighing 1.5 kg. From the knowledge acquired during the training, she referred the baby to the NCU and the mother was taught Kangaroo mother care techniques. The baby consequently put on weight and is now back to healthy levels after discharge from the NCU. Some of the beneficiaries reported to have been educated on essential newborn care especially through community durbars by community volunteers. But many of the mothers reported to have obtained education and information from nurses and volunteers who spend time with them during home visits. During the FGDs mothers reported to have been reached by project communication and social mobilization interventions like community durbars through volunteers and other community-based agents. Many of them noted the friendly and persistent attitude of the nurses and volunteers to educate them about home-based newborn care, and disease prevention. They expressed willingness to obtain more education and information and have nurses and volunteers spend more time with them during home visits. The majority of them who took part in the focus group discussions were able to recall the main messages regarding breast-feeding, hand washing, diarrhea management, danger signs in children and pregnant women. They confirmed that the information they received was very useful in their daily life and that they tried to put into practice the advice received. I always participate in durbars. They teach us general care of the child, general danger signs for newborn, hygiene, what to do during pregnancy, importance of facility delivery and postnatal care, hand washing before eating and after using toilet for both mothers and children, exclusive breastfeeding up to 6 months, baby feeding after 6 months, bathing and clothing the baby, Kangaroo mother care. After the durbar they give us drinks and we like that. (FGD participant, Upper East region) 41

42 In order to make it easy for mothers to put education knowledge into practice their mother in-law and husband were also taught newborn care good practices. The following is an illustration from a FGD with beneficiary mothers. Education during home visits helped us to learn many things. It was important that when nurses and volunteers came for home visits they insisted and met my mother-in-law and my husband and educated them also on danger signs for the baby, care of the cord, exclusive breastfeeding up to 6 months, how to position the baby at breast, when to immunize the baby and that both mothers and baby should sleep under mosquito net. This has helped me to practice and they support me and remind me what to do. My husband and in-laws are helping me to keep the household environment clean. (FGD participant, Northern region) Visual pictures and flip charts were easier for mothers to understand and retain the message being transmitted. They found pictures and flipcharts helpful. However, some mothers clearly preferred the case scenarios used in communicating the messages. In one case, the mother did not follow the education on newborn care and her baby ended up dying, and in another, a mother put into practice what she was thought and her baby survived. This was reported by one of the FGD mothers It was easier for us to have pictures and visual material that show how to practice and compare sick babies and healthy babies when newborn care is provided. Our child welfare book has at the back of its cover the danger signs for the baby we found it helpful and we carry it with us. (FGD participant, Upper East region) 42

43 Photo 3: A Focus Group Discussion session in the Northern Region, 22 nd January 2015 Objective 3: To ascertain the effectiveness of the project s evidence-based advocacy for issues related to newborn survival and national policy environment, The aim of the project to improve neonatal survival is very much aligned with the national child health policy as well as UNICEF s global mandate to improve child survival. A desk review of relevant policy documents showed that the objectives of the project were aligned with the Government of Ghana s Child Health Policy. The project is very much placed within UNICEF s global mandate to improve children s health. The project is operationally aligned with UNICEF s significant presence in northern Ghana including a field office in Tamale with technical and operational staff who provided close technical and monitoring support during the project. The project is also very well aligned with the EMBRACE model articulated in the Government of Japan s Global Health Policy both conceptually as well as operationally. First, the project in UER and NR aims to create linkages between facilities and community-based services through various actors at both community and facility levels. Second, the project is well positioned within the overall 43

44 continuum of care for maternal, newborn and child health and leverages UNICEF s investments for other parts of the continuum of care which support healthy childhood such as immunizations and prevention and treatment of the major child-killers such as malaria, diarrhea and pneumonia.. On seeking the degree of agreement or disagreement of national stakeholders and health partners on this issue, 12 out of 14 agreed or fully agreed that the intervention is relevant in terms of alignment of project objectives with the national strategy and the stakeholders priorities and needs, and also in terms of advocating for, and facilitating the introduction of newborn health into the national agenda of the MNCH Programming (Figure 4.1). However, one stakeholder had no opinion and another fully disagreed.. The project supported the processes leading up to the development and launch of the National Newborn Strategy and Action Plan. Specifically, the project supported MoH/GHS to organise 3 national level stakeholder meetings on newborn health. It also supported the development of a bottleneck analysis tool and decentralized the planning and monitoring of newborn health. As reported by key informants, the launch was very successful, and the advocacy and communication around it contributed to place newborn health on the national agenda A desk-review of the project document showed that the project supported (financial and technical) the MoH/GHS to organize workshops in Accra on newborn health issues. The aim of the meetings was to have a common understanding of the package of maternal and neonatal health interventions outlined in the current Child Health Policy and Strategy. The meeting took stock of the implementation status with a focus on neonatal health. The project has also supported the 44

45 development of the perinatal death audit tool. It has also advocated for the Ghana Pediatric Society to bring newborn care issues into the national agenda. In order to implement the National Newborn Care Strategy, all 10 regions are now provided with a newborn health focal person. On seeking the opinion from national decision-makers and health partners, 11 out of 14 agreed or fully agreed that the project has contributed to the policy direction of the National Newborn Strategy and Action Plan (Figure 4.2). However three national key informants had no opinion about this statement. National Key Informant interviews substantiate the remarkable efforts undertaken by GHS with support from UNICEF and funding from the Government of Japan, in the use of evidence-based interventions on newborn and child healthcare in the Northern and Upper East regions. GHS have built the capacities of primary, secondary and tertiary level healthcare providers in implementing newborn care activities. Testimonies from stakeholders, providers and beneficiaries were packaged and videotaped by the communication section of the MOH and used effectively to advocate for newborn health through media, and during important fora to bring newborn health high on the national agenda. The launching ceremony was attended by high profile stakeholders, donors and a pool of national media. There is, however, still room for additional visibility for the newborn health program in order to mobilize necessary resources to scale it up to a nationwide level and have it on a regional African agenda. The following are quotes from national key informants in support to the impact of the advocacy generated by the project. 45

46 I think that we have moved far, there is still room for improvement but we have come very far because 4 or 5 years ago we did not have much to talk about newborn. Now we have a newborn strategy. It was actually the Ministry of health and partners who recommended a newborn strategy and Ghana Health Service was tasked to have it developed. We now have the newborn indicators for performance monitoring. Now when we attend Ghana Health Service meetings of directors, Ministry of health, everybody is talking about newborn. (National key informant) The national newborn strategy was launched in July last year and that launch was a very big one. The advocacy and communication around it, was such that it went very far. We had parliamentarians, civil society, embassy representatives, some key media personnel and representatives from other sectors. Newborn became the talk of the town, Last year for example almost all the professional association groups, we call them medical superintendents, adopted a newborn theme for their annual general meeting. It was a very busy year for us; everybody wants us to come and speak on newborn at their general annual meeting. (National key informant) 46

47 Photo 1: Evidence of Advocacy for newborn during the launch of the newborn strategy by Minister of Health Furthermore, the Newborn Care (NBC) project through its advocacy efforts influenced the implementation of the newborn care strategy and the insertion of NBC indicators into the DHIMS2. Training packages, guidelines and protocols were developed to help implement the NBC strategy. There was an advocacy effort to keep newborn care upfront on the agenda of a number of health summits and various in-country regional and national meetings. One national key informant clearly stated, in the quote below, the important role UNICEF played in the advocacy campaign: I think in 2012, there was enlightened awareness and advocacy for newborn because we have not been able to significantly reduce newborn mortality. There was awareness raising and advocacy throughout the country. A national conference was held in In all the regions various stakeholders, regional directors and public health workers met and developed plans to implement newborn care strategy. Newborn care was on the agenda of a various meetings, WHO and UNICEF regional meetings and meeting in West Africa. Here UNICEF is doing a lot to try to keep newborn as part of the major health activities in the country. (National key informant) 47

48 Another key informant reported that there were lots of broad consultations before the project took off and also while the project was running. Following broad consultations, the government was at the table with other stakeholders to discuss issues of newborn care. The project succeeded in putting the government in the driving seat for newborn care. The advocacy efforts of UNICEF brought together many national stakeholders to participate in a meeting in Senegal where participating countries reviewed the draft global newborn action plan as illustrated in the quote below: I just remember a meeting we attended in Senegal concerning the newborn where UNICEF came back to the Ghana newborn working group. From there everything was about reviewing a global action plan for newborn which other countries were to adopt. Working together stakeholders formed working groups of all players in the newborn and child health space and I would say UNICEF played a very significant role in making it happen. (National key informant) Going down to the project districts, metallic signboards were erected at facilities and Government of Japan stickers affixed on equipment and supplies as well as doorways to NCU (See Photo 5). Opinions sought from key national decision-makers showed 11 out of 14 agreed or fully agreed that the project gave due importance on the government of Japan s visibility (Figure 4.14) but three (3) of them gave no opinion. They indicated that they have seen signboards of the Government of Japan at NCU. 48

49 Photo 5: Evidence of visibility of Government of Japan s contribution Objective 4: To draw lessons on the implementation capacity for national scale-up of the essential newborn care model through home-based early postnatal care. The Newborn Sub-Committee coordinates newborn care activities of all partners involved in implementing newborn interventions. Each region and district has a focal person for newborn care. The training modules have been adopted and are being rollout to other regions and the home visits are being integrated into the CHPS structure. 49