Evaluation of Integrated Management of Childhood Illnesses Initiative in the Republic of Moldova Years

Transcription

1 "Simple techniques save lives of children", Key informant Evaluation of Integrated Management of Childhood Illnesses Initiative in the Republic of Moldova Years Final Report Executed by: Center for Health Policies and Analysis in Health 99/1, V. Alecsandri st. Chisinau, MD 2012 Republic of Moldova Tel: Fax: Commissioned by: UNICEF Moldova 131, 31 August st. Chisinau, MD 2012 Republic of Moldova Tel: Fax: Coordinated by: Svetlana Stefanet, Chief of Equitable Access to Quality Services Programme Elena Laur, M&E Officer 1

2 Acknowledgements The IMCI evaluation has benefited from contributions of the Mother and Child Department of Ministry of Health, the national IMCI M&E unit, the National Center of Health Management and Family Medicine Centers and we would like to thank them for their efforts in providing access to national statistics, organising data collection process and bringing the report to a successful conclusion. We would also like to thank mothers and other caregivers of children, family physicians, primary health care nurses, health managers and key informants in providing their opinions necessary for the evaluation of the IMCI program in the Republic of Moldova. Finally, we thank UNICEF Moldova team for their excellent support throughout preparation of this report. PAS Center team 2

3 Acronyms ADD ARI DTP ECCD FP IMCI IMR KAP Left Bank LPA M&E MCH MICS MoH NCHM NGO NHIF NIS ORC PHC Right Bank SD SDC U5MR UNDP UNICEF WHO Acute Diarrheal Disease Acute Respiratory Infections Dyphteria, Tetanus, Pertussis Early Childhood and Care Development Family Physicians Integrated Management of Childhood Illnesses Infant Mortality rate Knowledge Attitudes and Practices Transnistrian region located on the left bank of Dniester River Local Public Authorities Monitoring and Evaluation Mother and Child Health Multiple Indicator Cluster Survey Ministry of Health National Centre for Health Management Non-Governmental Organisation National Health Insurance Fund Newly Independent States Oral Rehydration Corner Primary Health Care Republic of Moldova, Right bank of Dniester River Standard Deviation Swiss Agency for Development and Cooperation Under-5 mortality rate United Nations Development Program United Nations Fund for Children World Health Organization 3

4 Table of Contents Acknowledgements... 2 Acronyms... 3 Table of Contents... 4 Table of text boxes... 6 Table of tables... 6 Table of figures... 6 Executive Summary... 8 Evaluation context... 8 Goals of the evaluation... 8 Methods... 9 Key findings Conclusions Implications Background Purpose and Objectives of the Evaluation Methods Preparation phase Data sources Study setting and included populations Data collection phase Data entry and analysis Ethical considerations Qualitative component Limitations Context of the IMCI Implementation IMCI Program design Program coordination and main stakeholders IMCI program inputs Training Supervision system Informational support Program expenditures Perceptions of key informants and health managers regarding accomplishments and difficulties of IMCI implementation National IMCI M&E system Strengths of the M&E system Weaknesses of the M&E system IMCI program outputs, national M&E system Primary health care providers and IMCI Socio-demographics Inputs: Coverage with IMCI interventions Attitudes towards IMCI content Relevance: Attitudes of health care workers towards IMCI program implementation IMCI difficulties as perceived by family physicians Outputs: knowledge and practices of health providers Knowledge of major danger signs Knowledge of pneumonia management

5 Management of diarrhea Management of anemia Early stimulation for development Perceptions of health providers about outcomes and impact Coverage with key IMCI interventions, knowledge and practices of care givers of children age 0 to 5 years Sample demographics Child feeding practices Vitamin D Immunizations Knowledge of danger signs Diarrheea Basic IMCI knowledge indicators in caregivers Physician Counseling Mother s agenda Access to medical services Satisfaction with primary health services Achievement of original project outputs Contribution of IMCI to child health outcomes Immunization coverage Nutrition status Anemia Impact of IMCI implementation in the Republic of Moldova Infant Mortality Rate and and Under-5 Mortality Rates Equity and Vulnerability Equity and Child Mortality Equity focus in primary health practice Vulnerability as a barrier to achieving good health outcomes in children Opportunities for interventions The IMCI effect on equitable services Institutionalization and Sustainability Conclusions Lessons Learned Implications References Annex 1. Terms of Reference Annex 2. Questionnaires and Interview Guides

6 Table of text boxes Text box 1 Opinions of health providers about IMCI training Text box 2 Opinions of health providers about Mother s Agenda Text box 3 Strengths of IMCI Text box 4 IMCI relevance to Moldovan PHC practice Text box 5 Difficulties in IMCI reporting, as perceived by health providers Text box 6 Counseling for child feeding and early stimulation Text box 7 Perceptions about changed behaviors and practices Text box 8 Child feeding in first year of life Text box 9 Feeding practices in children over one year Text box 10 Knowledge about anemia prevention in care givers Text box 11 Opinions of care givers on immunizations Text box 12 Knowledge of danger signs by care givers Text box 13 Physician focus on counseling, as perceived by caregivers Text box 14 Opinions of caregivers about Mother s Agenda Text box 15 Opinions of caregivers about their interaction with family physicians Text box 16 Perceptions of key informants and health managers about IMCI impact Table of tables Table 1. Sample Distribution by sex, environment and geography Table 2 Distribution of qualitative interviews by sites Table 3 Comparison of basic indicators between caregivers accessing health services and household ECD 2009 survey Table 4 IMCI program effectiveness, compared to initial expected outputs Table of figures Figure 1 Logical framework of the IMCI program in the Republic of Moldova Figure 2 IMCI program costs, as registered by UNICEF Moldova, years Figure 3 Main national IMCI process indicators, year , Republic of Moldova, Right Bank only 28 Figure 4 Percent of children assessed through IMCI tool, including 5 danger signs and 5 basic signs, selected districts and average, years Figure 5 Percent of children assessed through IMCI tool, including 5 danger signs and 5 basic signs, total for the Right Bank, years Figure 6 Percent of children under two years who received vitamin D prophylaxis according to the standard Figure 7 Percent of hospitalized children based on danger signs and severe sickness referred by PHC workers Figure 8 Percent of children under 2 years who have received iron supplement for anemia prevention 30 Figure 9 Percent of children under 5 years who have received iron supplement for anemia prevention 31 Figure 10 Opinions of FPs about the most useful IMCI modules Figure 11 Level of satisfaction with the quality of IMCI implementation in the health institution Figure 12 Perceptions of health providers about difficulties and barriers in IMCI implementation Figure 13 Knowledge of danger signs, family doctors Figure 14 Knowledge of IMCI-specific signs of pneumonia, family doctors

7 Figure 15 Knowledge of IMCI signs of diarrhea, family doctors Figure 16 Knowledge of IMCI advice for child development at 6-12 months, family doctors Figure 17 Length of breastfeeding, health-seeking caregivers, excluding current breastfed babies Figure 18 Timing of introducing solids, health-seeking caregivers, subsample of caregivers of children 12 months -5 years Figure 19 Distribution of major nutrients in the 24 hours preceding survey, subsample of caregivers of babies aged 6-12 months Figure 20 Distribution of major nutrients in the 24 hours preceding survey, subsample of caregivers of babies aged over 12 months to 5 years Figure 21 Knowledge of danger signs and situations needing urgent medical care, health seeking caregivers Figure 22 Physician counseling on IMCI topics, health-seeking caregivers Figure 23 Degree of coverage of cost of drugs by health insurance Figure 24 Coverage rate with immunizations, trends for years , Republic of Moldova Figure 25 Percent of children breastfed at 6 months of age, trends for , Republic of Moldova Figure 26 Malnutrition in children under 1 year and under 5 years of age (per 1,000 children of this age), trends for , Republic of Moldova Figure 27 Anemia in children under 5 years of age (per 1,000 children of this age), trends for , Republic of Moldova Figure 28 Incidence and prevalence of anemia in children up to 18 years, per 10,000 inhabitants of this age Figure 29 Anemia prevalence in pregnant women who gave birth during that year, Right Bank only, in % Figure 30 Total and at home infant and under-five mortality rates (per 1,000 live births), trends for years , Republic of Moldova Figure 31 Under-five mortality rates by gender (per 1,000 live births), trends for years , Republic of Moldova Figure 32 Infant postneonatal mortality rate, years , Right Bank only, per 1,000 live births Figure 33 Percent of at home deaths in infant and under-five mortality rates, trends for years , Republic of Moldova Figure 34 Proportion of deaths within 24 hours of hospitalization of the total IMR, Right Bank only, in % Figure 35 Causes of under-5 mortality structure, trends for years , Republic of Moldova Figure 36 Under-5 mortality (total, at home, from respiratory system disorders, ARI, ADD, trauma and poisoning) , % to baseline (2000 = 100%) Figure 37 Infant mortality rate (total, at home, from respiratory system disorders, ARI, ADD, trauma and poisoning) , % to baseline (2000 = 100%)

8 Executive Summary Evaluation context The Republic of Moldova was among the first countries in the WHO European Region to implement the Integrated Management of Childhood Illness (IMCI) Initiative starting in 1998 as the most cost-efficient strategies of improvement of mother and child care. At the time, the first cause of under-5 child mortality was respiratory disorders and about 50% of causes of death were estimated to be preventable, while at-home mortality was high at over 20%. Additionally, children had higher prevalence of anemia, Vitamin-D deficiency and malnutrition compared to other Eastern European countries. The IMCI Program in the Republic of Moldova aimed to address leading causes of childhood deaths through improving case management skills of health care staff; strengthening health system performance and improving care giving practices in families and at the community level. The project goal came to support the realization of Moldova s Millennium Development Goals of reducing infant and under-five mortality rates. These goals are to be achieved through the implementation, with the support of international agencies and donor organizations, of the National Health Policy, National IMCI Strategy, Health System Development Strategy and National Development Strategy. The IMCI initiative in the Republic of Moldova was implemented in three phases. Phase 1: Program adaptation and introduction ( ) Phase 2: Program piloting ( ) Phase 3: Program scale-up ( ) After an adaptation and pilot phases until 2002, the years were the years with the most intensive IMCI program implementation for the Right Bank and for the Left Bank. The essentials of IMCI are implemented countrywide, including the Transnistrian region, with support of UNICEF, and its cost-sharing partners, WHO, SDC and others. The initial two phases received technical and financial support of WHO Office for Europe, WHO Moldova and UNICEF. Starting with year 2003, Swiss Agency for Development and Cooperation (SDC) co-financed UNICEF for support of scale-up program phase. UNICEF ensured programmatic and financial management of all IMCI activities, while the Mother and Child Health (MCH) Department at the Ministry of Health ensured overall coordination of implementation. Since 2007 UNICEF and its implementing partners have gradually decreased their technical and financial support to IMCI activities. Mother and Child Department of the Ministry of Health and the M&E Unit of the Mother and Child Institute have taken over full oversight and monitoring functions of the program in In November 2010, the Centre for Health Policies and Studies (PAS Centre) was contracted by UNICEF Moldova to conduct an end-of-program evaluation of IMCI implementation in the country. The primary intent of the evaluation is to see if IMCI had an impact, and especially what remaining barriers need to be addressed in order to ensure its sustainability, acceptance and integration. It is expected that the evaluation results will be used by the government, health authorities as well as by external partners in planning of actions aimed at further improvements in the delivery of health services for children under five years of age and families in the country. Goals of the evaluation The main goals of the evaluation were: a) To assess the relevance of Project outcomes, the effectiveness and efficiency by which IMCI project outcomes are being achieved, their sustainability and contribution to overall policy framework in the context of Mother and Child Health Care system 8

9 b) To conduct impact evaluation on child health, taking into consideration initial situation (baseline), the timing of the interventions and how long it might take from intervention e.g. training of medical staff or other stakeholders to see the effect on child health and other beneficiaries with particular attention to gender issues and reaching the most vulnerable ones when appropriate. Methods The evaluation revolves around the evaluation criteria stipulated above: (i) relevance, (ii) efficiency, (iii) effectiveness, (iv) impact, (v) equity and (vi) sustainability. The initial design was a formal summative evaluation to rely on analysis and synthesis of existing data sources and complement existing quantitative data with qualitative research. However, in the process of refining the evaluation methodology and during the desk review phase we found limitations in data availability and quality and additionally included quantitative survey of health providers and care givers, outcomes in order to measure knowledge and behaviors of these categories. The data sources for this evaluation were: (1) national routine statistics for data on infant mortality rates and under-5 child mortality rates, immunization coverage, anemia and malnutrition rates in children under five years; (2) national IMCI M&E reports for program output and process indicators; (3) survey data of primary health care providers; (4) survey data of health seeking caregivers of children of 0-5 years; (5) structured interviews with city and rayon health managers responsible for IMCI in their administrative unit; (6) structured interviews with key informants - national stakeholders involved in IMCI implementation and coordination. A representative sample of 10 administrative units was included in the evaluation. The target populations were (1) health managers of raion or municipal Family Medicine Centers; (2) primary health care providers from urban and rural family medicine or health centers; (3) main care givers of children of 0 to 5 years (mothers, fathers and grandmothers); (4) key informants. For data collection a face-to-face interview was used for the survey of PHC physicians and caregivers and structured in-depth interviews and a focus group discussion were used for the qualitative data collection form health managers, key informants, PHC providers and caregivers. SPSS 13 software was used for data entry and analysis. After entering all data, the database was checked for accuracy based on filter questions, transition questions and the internal logic of the questionnaire. Data analysis included frequency reporting. The structured interviews were recorded, transcribed and coded. The major limitations are: (1) given the budget limits and the change in evaluation methods, the sample size for the survey component was small with a large confidence interval. Because of small sample size, breakdowns by rural/urban, socio-economic status were not possible. Yet, since outcome results show high percentages for major indicators of knowledge, the large confidence interval does not pose an interpretation problem in most instances; (2) evaluation is observational, given the lack of a formal baseline survey to compare data against. 9

10 Key findings 1. The inputs of the IMCI program in Moldova (training and supervision components) were welldesigned, high quality and high-coverage. Some 90% of family physicians (FPs) on the Right Bank 95% on the Left Bank were covered with standard training. For nurses, the coverage is lower at 41% on the Right Bank and 71% on the Left Bank. Physicians acknowledged a high level of satisfaction with the quality and relevance of training and coverage with continuous supervision system. 2. Both physicians and caregivers expressed a high level of awareness and use of Mother s agenda, a tool used for increasing caregivers knowledge about child feeding and development, knowledge of danger signs, immunizations and trauma prevention, but noted its limited supply at present. The total number of 200,000 copies of Mother s Agenda printed with UNICEF support throughout the ten years period, seems to be highly insufficient, since the current total number of children of 0-5 years on the Right Bank is 191, The expected program outputs have been achieved and exceeded. Higher proportion of PHC workers were covered with training (90% compared to original 60%), and higher proportion of caregivers received Mother s Agenda (72% compared to original 60%). 4. Knowledge of IMCI content by physicians is good: 90% of them were able to name 3-4 out of 4 listed danger signs; 94% have mentioned IMCI signs for pneumonia and 95% the correct antibiotic of choice in treating pneumonia; 95% named at least 3 signs of diarrhea, 99% screening for anemia based on palm paleness and 61% named 3 or more early stimulation techniques. 5. High proportions of caregivers have mentioned that FPs have counseled them for immunizations (85.3%), child feeding (82%) and danger signs (77%), and lower proportion have talked to their doctor about child development (57%) and trauma prevention (60%). Over two-thirds (72%) have received mother s agenda. 6. IMCI process indicators as reported by the national M&E system in 2010 include: a. 80% of children assessed through IMCI patient evaluation tool b. 8% of children identified with danger signs c. 24% of children hospitalized (declining from 33% in 2008) d. 17% of children identified with anemia based on palm paleness e. 92% of children receiving Vitamin D until the age of 2 years 7. Nationwide outcomes have shown mixed results: a. Immunization coverage went up until 2007 and then declined to a decade low 93% for dyhpteria, tetanus, pertussis (DTP), but is still over 90% for all immunizations, despite an increasing caregiver active opposition to immunizations. b. Malnutrition rate for under-one-year has significantly declined from 80 to 28 per 1,000 children under one year and for under-5 from 23 to 11 per 1,000 children under 5 years (comparison years 2000, 2010). c. Anemia rates have increased from 74.1 per 1,000 children of 0-5 years to in 2010, possibly due to better screening as well, but the proportion of children receiving iron supplement is low (20% average based on six selected sites reporting correctly in 2010) 8. Practices of caregivers who are in contact with PHC physicians have shown good levels: a. Some 95% mothers breastfed their children since birth, with an average length of breastfeeding of 11 months. The majority (87%) were breastfed at least 6 months and 36.3% over 12 months. The average age of introducing solid foods was 5.5 months. b. Some 88% were able to mention at least two danger signs (compared to 73.0% in MICS 2000 and 81.0% in ECD 2009). 10

11 c. In children over 12 months, only 60.7% received all four main groups of foods in the past 24 hours (meat or fish, vegetables and fruits, dairy products and grain), mostly due to poor financial status rather than knowledge about the needs. d. The vast majority (95.3%) have provided their children with Vitamin D, with an average length of 15 months. e. The majority (86.3%) received all immunizations on time and of those who did not, the major reason was a delay due to sickness (68%), while 25% are strongly against or skeptical about immunizations. 9. Impact: the under-five mortality rate has seen a significant reduction from 23.2 per 1,000 live births in 2000 to 13.6 in 2010, with steepest reductions registered in the years , when most IMCI training and other activities have been implemented in country districts. The IMR has also seen a gradual and stable reduction from 18.3 in 2000 to 11.7 in 2010, even with the adoption of the WHO definition of lifebirth in 2008 that has not shown a large increase in the following year. It is difficult to evaluate the extent of IMCI contribution to reductions in IMR and U5MR given several other major processes that have contributed to decreasing mortality rates. a. If compared to year 2000 as 100% baseline, the categories that have seen the most percent reduction to baseline in 2010 were U5MR due to acute respiratory infections and other respiratory system disorders (by 75% in 2009 and 50% in 2010 for ARI and 62% for other respiratory disorders) and acute diarrheal diseases by 60%. Since these were the main focus of IMCI strategy, these specific U5MR reductions are most likely attributable to IMCI strategy, especially since a less significant reduction in mortality due to trauma and poisoning was registered (only 12%) (not an initial IMCI focus). b. In the case of IMR, a 50% reduction in the total IMR rate was registered in 2010 compared to 2000, a significant reduction by 55.6% by 2010 was registered in rates of respiratory system disorders and a 50% reduction for at-home deaths were registered in 2009, probably attributable to IMCI strategy as well. c. Postneonatal IMR, an indicator more specific to IMCI has also registered a significant reduction from 7.6 in 2000 to 4.3 per 1,000 live births in d. The proportion of at-home deaths of the total IMR and U5MR have registered a decrease starting with year Both at-home IMR and U5MR have registered a steady decrease until 2009 (for U5MR from 25.0% in 2000 to 20.1% in 2009 and for IMR from 19.7% to 14.9%) and important increases in 2010, that need to be further investigated and explained. 10. Equity: Mother and Child Health Equity Analysis conducted in 2010 concluded that whereas overall child mortality rate decreases, the remaining rates disproportionately affect vulnerable children from rural areas, lower SE quintiles and Southern region. The same analysis reported a moderate level of inequity in nutritional status and anemia rates in children. a. The IMCI strategy implemented in Moldova has lead to universal coverage of children with basic benefit package. Yet, of the caregivers who had a sickness episode and were prescribed medicines, in only 25% the health insurance covered fully the cost of prescribed drugs, according to care givers. In rural areas and raion centers three times more people relied only on the drugs covered by basic benefit package only (10% in cities, 32% in raion centers and 37% in rural areas). b. The consensus of key informants and health managers was that IMCI strategy provides equitable access to health care, especially by providing a universal coverage with basic benefits package. Other IMCI strategies ensured well equitable access, such as promotion of breastfeeding, universal coverage with caregiver education. 11

12 11. Relevance of IMCI strategy is perceived differently by health providers. PHC physicians from rural areas, PHC who have been retrained from internal medicine and nurses and younger health providers find it highly useful, while PHC nearing the retirement age, former pediatricians and those living in Chisinau, Balti and some district centers perceived the program as too basic and underestimating medical practice. 12. Barriers: Only half of physicians are satisfied with IMCI implementation, the main impeding factors being an overburden of paperwork and reporting and the list containing drugs with low acceptance or restrictions in the choice of antibiotics. 13. The national M&E system has been successfully instituted nationwide, but currently needs major revisions in decreasing redundancy of the list of indicators and better definitions, as well as decreasing the burden on health providers. 14. Institutionalization and sustainability have been a contionuous focus from design and throughout implementation. The coordinating function of whole program is implemented by a national coordinator under the Mother and Child Department of the Ministry of Health; the national monitoring and evaluation of the IMCI activities are performed by the M&E unit of Mother and Child Health Institute; the IMCI trainings are credited and accounted in the Continuous Medical Education system; the trainees are certified and deployed within the system by Ministry orders; the venues are offered by the system; the IMCI is included in curriculum for medical students (both physicians and nurses); the IMCI materials (Mothers Agenda and Child Medical Records) are official medical forms, institutionalized by the system. Conclusions The overarching conclusion is that IMCI has worked well in the Republic of Moldova. The IMCI strategy has had impact on decreasing under-5 child mortality and infant mortality rates, although difficult to evaluate the extent of contribution to this significant reduction because of parallel interventions. Yet, the IMCI strategy has probably had an impact on changing the child mortality structure and on significant decrease in U5MR and IMR due to acute respiratory diseases and acute diarrheal diseases and decrease in postneonatal IMR. The area where IMCI had less significant and sustained effect is the percentage of at-home deaths and within 24 hours of hospitalization in children and mortality due to unintentional injury and poisoning. For outcomes, the IMCI program has been effective at improving knowledge of health workers and changing attitudes towards IMCI child standardized assessment. Despite initial perception of a too basic content for physicians, the IMCI practice managed to convince a significant proportion of health providers that simple techniques are effective in saving lives and improving health outcomes. Preventive measures of screening for stunting and anemia, prophylaxis with Vitamin D, prophylaxis of helminthiasis have increased over time. The areas where results have not been achieved yet are anemia prevention and treatment. This evaluation found that IMCI program in Moldova is a strong, well-designed, well-implemented highquality and high-impact program, with relatively modest financial external contributions, as compared to its outcomes and impact. The IMCI program has achieved and exceeded its outputs of coverage with training (except PHC nurses, as only nurses from health facilities that are not staffed with FPs have been trained), IEC and supervision of health providers and coverage with information materials of caregivers. The program has also been effective in changing knowledge and practices of caregivers as well, especially in the area of child feeding and knowledge of danger signs. The overlooked area where little progress has been achieved is counseling for early stimulation and development. 12

13 The IMCI strategy is perceived by health managers, key informants and health providers to have increased equitable access to health care. However, the mortality statistics still show a disproportionate contribution of the poor, rural and residents of Southern regions to child mortality. IMCI aimed at increasing knowledge and access to health care, but these are not the only determinants of health outcomes, the economic situation and educational levels are important contributors as well and structural interventions are needed as well. In terms of institutionalization, the program has had a good strategy from the outset and has made important savings by being implemented within the national health system and by the health system employees and is a good example of institutionalization for other donor-supported programs. Some areas still need external technical assistance and support, such as revising the current national IMCI M&E system. One unintended consequence that lead to increasing service provider resistance towards the whole IMCI strategy has been the reporting system that has added a significant writing burden on health providers. There is also room for improvement for a better integration of IMCI standards in the current national standards and within the medical documentation and sustainability of the informational and educational components. Implications 1. To strengthen IMCI activities in the areas where changes have not yet produced a sustained effect: a. For effecting at-home mortality, to continue and increase focus on danger signs awareness, particularly in the most vulnerable families. The initiative of the local IMCI coordinator in Cimislia to print danger signs as a poster and give it to providers to be posted in their homes could be scaled-up to areas with higher mortality rates than the country average. The second initiative of a health manager, to develop a list of children at higher risk for morbidity and mortality and setting responsible family physicians and social assistants could also be considered as a model for scale-up. b. An additional in-depth study on anemia control in children might be required, to better understand the increasing trend in anemia rates in children. Based on the qualitative findings of this study, we would recommend to NHIF to look into possibilities to include a more acceptable drug in the basic package, and for health providers to intensify advice on the need of sufficient protein and vegetable intake and supervise the quality of screening and iron-supplement prescription by PHC workers. Flour fortification with iron could be considered as an option as well. c. For strengthening counseling for development, to revise the current module to see if it is effective, devise a standard algorithm for counseling for development and provide local refresher courses to PHC workers. In order to increase time available for counseling, decrease the reporting burden from PHC workers. 2. To make a revision to IMCI provisions based on its relevance to national protocols and DCM classification, revise the current list of drugs and revise the current IMCI assessment form to make it more useful and user-friendly for family physicians and more integrated with the other requirements. 13

14 3. To revise the current M&E reporting requirements, namely to revise the overall list of indicators, improve and standardize their definitions, revise data collection forms and reduce the reporting burden at the health provider level, by decreasing the periodicity of and complexity of reporting. 4. Explore a more sustainable model for institutionalization of printing costs of informational materials for care givers, as the model where local health authorities are responsible for its printing has not worked thus far. 5. To continue in-service training for PHC nurses through standard 96-hour training and maintain high quality of IMCI training in medical. Additional actions to enhance effectiveness of nurses might include revision of their job description, performance-based payments, additional training on helath promotion and other as found appropriate by national stakeholders. 6. To provide IMCI orientation training to emergency room health staff 14

15 Background The Republic of Moldova was among the first countries in the WHO European Region to implement The Integrated Management of Childhood Illness (IMCI) Initiative starting in The IMCI is essential health care technologies that rely on case detection through utilisation of a limited number of simple clinical signs and empirical treatment regimens. The signs are based on expert clinical opinion and research results; the treatments are developed according to action-oriented classifications rather than exact diagnosis. IMCI had been recommended to Moldovan Government and Ministry of Health by the WHO, UNICEF, and World Bank as the most cost-efficient strategies of improvement of mother and child care with a considerable potential to accelerate the processes of social and economic development of the country. The IMCI Program in the Republic of Moldova aimed to address leading causes of childhood deaths through improving case management skills of health care staff; strengthening health system performance and improving care giving practices in families and at the community level. Currently the essentials of IMCI are implemented countrywide, including the Transnistrian region, with support of UNICEF and its cost-sharing partners. After an adaptation and pilot phases until 2002, the years were the years with the most intensive IMCI program implementation for the Right Bank and for the Left Bank. Since 2007 UNICEF and its implementing partners have gradually decreased their technical and financial support to IMCI activities. Mother and Child Department of the Ministry of Health and the M&E Unit of the Mother and Child Institute have taken over full oversight and monitoring functions of the program in In November 2010, the Centre for Health Policies and Studies (PAS Centre) was contracted by UNICEF Moldova to conduct an end-of-program evaluation of IMCI implementation in the country. The primary intent of the evaluation is to see if IMCI had an impact, and especially what remaining barriers need to be addressed in order to ensure its sustainability, acceptance and integration. It is expected that the evaluation results will be used by the government, health authorities as well as by external partners in planning of actions aimed at further improvements in the delivery of health services for children under five years of age and families in the country. Purpose and Objectives of the Evaluation The evaluation of the IMCI project will serve the following main purposes, as per initial Terms of Reference (Annex 1): c) To assess the relevance of Project outcomes, the effectiveness and efficiency by which IMCI project outcomes are being achieved, their sustainability and contribution to overall policy framework in the context of Mother and Child Health Care system d) To conduct impact evaluation on child health, taking into consideration initial situation (baseline), the timing of the interventions and how long it might take from intervention e.g. training of medical staff or other stakeholders to see the effect on child health and other beneficiaries with particular attention to gender issues and reaching the most vulnerable ones when appropriate. Scope of the evaluation: to analyze IMCI project interventions in the wider context of Mother and Child Health Care system. 15

16 Methods The evaluation revolves around the evaluation criteria stipulated in Terms of Reference (Annex 1) : (i) relevance, (ii) efficiency, (iii) effectiveness, (iv) impact, (v) equity and (vi) sustainability. They were all deemed appropriate for the purpose of this evaluation. Since IMCI was not an emergency nor complex humanitarian program, the additional criteria namely coverage, coordination and coherence, were addressed in a lesser extent. Evaluation design development was a joint process between UNICEF and the PAS Center. The initial design as per Terms of Reference (Annex 1) was a formal summative evaluation to rely on analysis and synthesis of existing data sources and complement existing quantitative data with qualitative research, in order to answer the why questions. However, in the process of refining the evaluation methodology and during the desk review of existing routine statistics and IMCI statistics, given the limitations in data availability and quality, the evaluation team suggested a change in methods, to include an additional quantitative survey of health providers and care givers, in order to measure outcomes (knowledge and practices). The approach for this evaluation has been based on the principles of participation and cooperation. Throughout all stages of the evaluation, the evaluation team has liaised with the different key players, such as UNICEF Office, key informants, Ministry of Health officials, local health authorities. The implementation of the evaluation and data collection by a local consulting team had the advantage of understanding the context of program implementation, local customs, personal interaction and gender roles, disabilities, age and ethnicities. Stakeholders were mainly involved into the evaluation process as sources for data collection and analysis. Upon finalization of the Evaluation Report, stakeholders will further follow up to address key findings and recommendations to take actions while planning the program strategy for child care. Human-rights based approach was included in the evaluation through an analysis of the right holders well-being (child health outcomes and well-being in the impact analysis), and duty bearers (the evaluation of the role of the Ministry of Health and health managers in implementation of the program, the quality of service provision and care to children by service providers and care givers). Gender equality was considered in analyzing impact indicators and is presented in gender breakdown for under-five mortality rate. Preparation phase During the preparation phase, the evaluation team started with studying the available information and documentation related to the IMCI project. The desk review included a comprehensive review of the available information on IMCI project available to date. The desk review was oriented at: Analysis of existing project documents and policies to assess the project design and implementation and related governmental support to IMCI program at different levels. Documentary sources included program activity reports and interim evaluation reports, UNICEF planning and programming documents, relevant and related professional publications and documentation. Analysis of routine statistics to evaluate the trends in infant mortality and under-5 mortality by cause, environment, family vulnerability, deaths at home in the period ; Comparisons of survey data with Early Childhood Development Knowledge Attitudes and Practices Household Survey conducted in 2003 and 2009 (ECD 2003 and 2009 KAP), MICS 2000 of outcome indicators in households regarding major health knowledge and behaviors (nutrition, danger signs). 16

17 Analysis of quantitative data reported in IMCI M&E framework at the national level. After the initial desk review, the team prepared a set of evaluation tools, such as guides for structured interviews/focus group discussions and questionnaires for health providers and caregivers. (Annex 2) Data sources National routine statistics for data on infant mortality rates and under-5 child mortality rates, immunization coverage, anemia and malnutrition rates in children under five years National IMCI M&E reports for program output and process indicators Survey data of primary health care providers Survey data of health seeking caregivers of children of 0-5 years Structured interviews with city and rayon health managers responsible for IMCI in their administrative unit Structured interviews with key informants: national stakeholders involved in IMCI implementation and coordination Study setting and included populations A sample of 10 administrative units was included in the evaluation. We have randomly selected two raions 1 per each region as follows: Northern region: Soldanesti, Riscani Central region: Orhei, Ungheni Southern region: Stefan Voda, Ceadir Lunga Transnistrian region: Grigoriopol Cities of Chisinau, Balti and Bender In each of the raions and cities, the categories involved in the evaluation were (1) health managers of raion or municipal Family Medicine Centers, (2) primary health care providers from urban and rural family medicine or health centers and (3) main care givers of children of 0 to 5 years (mothers, fathers and grandmothers). In addition, the evaluation team has conducted structured interviews with key informants (national IMCI coordinators (past and present), counterparts involved in the design and implementation of IMCI at the national and local levels, trainers and local facilitators or supervisors. Health providers: the final sample size constituted 127 FPs) with a confidence level 95% and a confidence interval of 8% (source population 1,900 family physicians). The sample included family physicians from the raion level family medicine center and one or two health centers in rural areas in that district. The sampling approach was a time-location sample of family doctors being at the time of data collection in the family medicine center or health center of the geographic sites. The gender distribution of FPs was 93% females, compared to 78% females in the total number of FPs (Association of Famiy Physicians of Moldova, 2010). Nurses were not included in the quantitative part of the evaluation due to evaluation budget limitations. Health seeking care givers: A total 190 caregivers were included in the survey, with a confidence level 95% and confidence interval of 7% (source population caregivers of 191,000 children of age 0-5 years). The sampling strategy included time-location sampling by randomly picking offices of family doctors and approaching all patients waiting in line to family doctors and recruiting all those that were care givers of children aged 0-5 years. 1 Raion is the local name of the administrative unit in the Republic of Moldova 17

18 Table 1. Sample Distribution by sex, rural and urban and Banks Total Males Females Rural Urban (raion centers and main cities) Right Bank Left Bank Health providers Caregivers Data collection phase Data collection tools were questionnaires developed specifically for the purpose of this evaluation. The questionnaires was available in Romanian and Russian, both versions were pre-tested on a sample of 20 persons. (Annex 2) Prior to data collection the interviewers were trained in the following areas: stages of survey implementation; structure and content of the questionnaire; details of the questionnaire for face-toface interviews; selection of respondents; data quality and validation; ethics and deontology of field interviewers. Interviewers trained in data collection for this evaluation administered the questionnaires during face-to-face interviews, after obtaining informed consent. The interviews took place on site of health premises through face-to-face interview. The questionnaires were filled in with full observance of anonymity and confidentiality requirements. Prior to data entry, the supervisor checked all the filled in questionnaires for errors and consistency. Data collection phase was conducted in February-May Data entry and analysis SPSS 13 software was used for data entry and analysis. After entering all data, the database was checked for accuracy based on filter questions, transition questions and the internal logic of the questionnaire. Data analysis included frequency reporting. Ethical considerations The evaluation included ethical safeguards, including protection of the confidentiality, dignity, rights and welfare of human subjects, particularly children, and respect for the values of the beneficiary communities. Voluntary participation based on informed consent was ensured. The questionnaire was anonymous and confidential. Qualitative component The qualitative component of the evaluation included structured interviews with the following populations. (i) key informants (ii) health managers (iii) health workers: family physicians from rural and urban areas; family nurses from rural and urban area and others (including Transnistrian region); (iv) care givers of children 0-5 years (mothers, fathers and grandmothers); Data collection was via semi-structured interviews, facilitated by topic guides, and designed to explore participants accounts. (Annex 2) Key areas of interview discussion included: Key informants: role in IMCI implementation, opinions about IMCI relevance and effectiveness and quality of implementation, perceptions about outcomes and impact, effect of IMCI on equity 18

19 Health managers: coverage with IMCI training and supervision, opinions about IMCI relevance and effectiveness and quality of implementation, perceptions about outcomes and impact, equity in access to health care for 0-5 years Health providers: coverage with IMCI training and supervision, opinions about IMCI relevance and effectiveness and quality of implementation, perceptions about difficulties, equity in access to health care for 0-5 years Care givers: practices of child feeding and medical advice for feeding, early stimulation practices and advice for early stimulation, coverage with interventions (Mother s Agenda, immunizations, vitamin D prophylaxis, anemia prevention, and satisfaction with physician-patient interaction), case management in the most recent sickness episode. Table 2 Distribution of qualitative interviews by sites Qualitative survey # people Site Key informants 9 Chisinau and Bender (urban) Health managers 14 All sites in the sample (cities and raion centers) PHC physicians 9 Chisinau, Bender (cities) Straseni (district centers) Lapusna, Gura Galbenei, Truseni (villages) Nurses 5 Chisinau, Bender (district centers) Gura Galbenei, Truseni (villages) Caregivers 24 Bender (city), Chisinau (capital city FG with 6 people) Singerei, Straseni (district centers) Lapusna, Gura Galbenei, Truseni, rural areas in Nisporeni, Cantemir, Telenesti raions (villages) All interviews were audio-recorded with informed consent. Interviews took place at the premises of primary health facilities or the PAS Center and lasted between 30 and 90 minutes. All interviews were transcribed verbatim, translated into English, coded initially for emerging core descriptive content, with coding further refined in an iterative process of data coding, charting and interpretation. Limitations Given the budget limits and the change in evaluation methods, the sample size was small with a large confidence interval. Because of small sample size, breakdowns by environment were not possible. Yet, since outcome results show high proportions for major indicators of knowledge, the large confidence interval does not pose an interpretation problem in most instances. Because of the budgetary constraints nurses were not included in the quantitative part of the evaluation, despite their important role in implementing the third component of IMCI strategy. The evaluation is observational, given the lack of a formal baseline survey to compare data against. Since the program was implemented nationwide, there were no comparison sites, therefore the impact indicators are compared before and after the interventions and trends are presented. The evaluation team has considered with caution the attribution of effect of IMCI program and the interpretations of impact findings are presented with this limitation. 19

20 Context of the IMCI Implementation After regaining its independence in 1991 after the break-up of Soviet Union, the Republic of Moldova has embarked on an ambitious health reform process, aiming at changing the health system architecture and a shift towards primary health care based on family medicine concept. In 1998, the Ministry of Health took the decision to introduce the IMCI strategy in the Republic of Moldova and several factors determined the national decision-makers to adopt IMCI concept as relevant for Moldova. At the time, the first cause of under-5 child mortality was respiratory disorders mainly due to pneumonia, a preventable cause of death. The infant mortality and under-5 mortality also included trauma and poisoning and acute diarrhea and about 50% of causes of death were estimated to be preventable. Athome mortality and mortality in the hospital within the first 24 hours was high, at over 20%, also mainly determined by pneumonia and delayed addressability of caregivers to health care services. Additionally, children had higher prevalence of anemia, Vitamin-D deficiency and malnutrition compared to other Eastern European countries. After introduction of family medicine through primary health care reform, the primary health care physicians, a half of who were formerly internal medicine specialists, became responsible for clinical care of pregnant women and children and needed rapid and easy tools to assess, treat and timely refer to appropriate level of care the children of 0 to 5 years. In addition, many rural areas and districts lacked PHC doctors and available nurses needed to get the necessary set of clinical skills as well. In parallel with primary health care reform, the Government has implemented a nationwide Mandatory Health Insurance System. Due to UNICEF advocacy efforts, from the outset in 2004 the NHIF granted universal coverage of free-of-charge health services for pregnant women and children up to 5 years (as part of IMI strategy). IMCI Program design The overall Project goal was to decrease Infant and child under 5 years old mortality and to improve the child health and development in Republic of Moldova by ensuring improved health care services and improving family and community practices. The project goal came to support the realization of Moldova s Millennium Development Goals of reducing infant and under-five mortality rates. These goals are to be achieved through the implementation, with the support of international agencies and donor organizations, of the National Health Policy, National IMCI Strategy, Health System Development Strategy and National Development Strategy. The original objectives were formulated as follows: 1. Reinforce health care providers skills and care practices in order to improve health outcomes and quality of care, with special emphasis on primary care; 2. Develop and promote strategies to ensure full continuity of care and consistency of practices throughout the health system; 3. Develop and promote strategies to improve family and community practices in care-seeking behaviour, home management of common diseases and nutrition The Project s strategy was to upgrade the practices of both healthcare providers and managers of primary healthcare services to deliver integrated child care services and improve care givers/families practices related to early care and development of children. The expected outputs, as formulated in project documents were the following: 20

21 1. At least 60% percent of primary health care workers (family doctors and nurses) in the Republic of Moldova, including the Transnistrian region will possess knowledge and skills to provide basic IMCI services of good quality, including counselling and support to parents with young children; 2. Health care managers in the Republic of Moldova, including the Transnistrian region will possess knowledge and skills to supervise and evaluate the accessibility and quality of IMCI services; 3. At least 50 percent of healthcare providers will be evaluated with regards to utilization of IMCI practices; 4. At least 60 percent of families with children under 5 years-old will receive materials containing essential information on IMCI topics during routine visits to healthcare providers and through Wellchild offices and benefit from qualified advice. Target groups of the program were primary health care workers, PHC managers, mother and child coordinators, chief of pediatric hospital departments, faculty of medical university, medical colleges and continuing medical education. The ultimate beneficiaries were children aged 7 days to 5 years and their caregivers. Figure 1 Logical framework of the IMCI program in the Republic of Moldova 21

22 Program coordination and main stakeholders The Ministry of Health (MoH) had an overall coordination role in the project and ensured quality monitoring of planned activities. MoH had also an important role in ensuring the sustainability of the project by providing essential drugs for IMCI, and ensuring integration of the training module into the pre-service training of health care providers at the medical university and college level. A national coordinator was designated by MoH to supervise all IMCI activities. The national IMCI Coordinator was responsible for planning of the training courses, facilitating the IMCI training courses for PHC providers and supervisors, organizing follow-up after training visits, preparing the evaluation report, and presenting the results of the evaluation to MoH. The IMCI component of the project was directly implemented by rayon health authorities under the supervision of the MoH. Local health authorities, and more specifically the chiefs of rayon hospitals, were responsible for making all the logistical and administrative arrangements related to the training courses arranging for meals, lodging, training facilities, mobilizing health care workers and supervision visits to primary health care facilities. The initial two phases received technical and financial support of WHO Office for Europe, WHO Moldova and UNICEF. Starting with year 2003, Swiss Agency for Development and Cooperation (SDC) co-financed UNICEF for support of scale-up program phase SDC contribution was instrumental in ensuring the implementation National IMCI Strategy. More specifically, SDC assistance contributed to geographical expansion of IMCI training to ensure nationwide coverage, thus complementing the work supported by UNICEF and WHO in the area of Mother and Child Health. Additional partners in the initial phase include Health Investment Fund and non-profit organization Amici dei Bambini. UNICEF ensured programmatic and financial management of all IMCI activities and ensured coordination of SDC support to the one provided by UNICEF and WHO in the area of mother and child health, local UNICEF staff were undertaking systematic field visits to project sites to monitor the implementation of SDC supported activities and made recommendations or suggest corrective actions to its main partners, prepared progress and financial implementation reports. Overall, the coordination ensured timely and effective implementation of the project, despite some challenges along the way, such as replacements of national coordinators, turnover in the teams on trainers. Although these challenges were overcome, and the training was implemented in time and with good results, it influenced the morale of the teams involved, and required additional efforts and attention. IMCI program inputs The implementation of the program has evolved in three phases: Phase 1: Program adaptation and introduction ( ) Phase 2: Program piloting ( ) Phase 3: Program scale-up ( ) During phase 1, a national working group oversaw and adapted IMCI training curriculum and training materials, developed job aids and mother s agenda, reviewed and included the list of IMCI drugs in the 22

23 List of Essential Medicines. The adapted IMCI package was reviewed and received approval from WHO Euro office. Phase 2 included training of the national team of trainers and initial training of PHC workers in the pilot district (Hincesti), national supervisors, and adaptation of training curriculum to add the module Care for Development, revision and printing of Mother s Agenda and Parents Guide. Phase 3 included the following scale-up activities: Capacity building of health care providers and managers in selected districts through the provision of training courses, technical assistance, and supervision. The training courses promoted cost-effective and evidence-based technologies promoted by WHO & UNICEF in child health programs. Primary health care workers received training on a number of issues related to basic MCH issues: child growth and development monitoring, integrated IMCI. Health care managers responsible for the organization of mother and child health services received training on use of supervision and evaluation tools to be able to appropriately plan, implement and evaluate activities in the area of IMCI. Information and communication at the individual and family level in the area of IMCI. The IMCI strategy focused on how to reach children where they live and on how to address knowledge gaps at the family and community levels regarding children health, recognition of illness, home care and appropriate care seeking. The strategy emphasized the importance of comprehensive development and care for children, as well as the need to reach the most vulnerable children. Training As of the end of 2010, a total 1,708 family physicians received the IMCI course, from a total 1,888 total number of FPs working in PHC on the Right Bank, representing 90% coverage with training on the Right Bank. Of the total number of 5,415 nurses working in PHC in 2010, a total 2,200 were trained in IMCI, totaling to 41% coverage. Additionally, an estimated 150 other health staff, including teaching faculty from the N Testemitanu State Medical and Pharmaceutical University and Medical College, as well as all raion-level health managers were trained in IMCI. On the Left Bank a total 183 physicians, representing 94.8% of the total number and 209 nurses representing 71% were trained in the years In addition, 17 teaching staff from local university and 2 medical colleges from the region have been trained. Supervision system The IMCI approach has included from its initial design and through implementation a robust supervision system through a network of 40 IMCI supervisor trained at the National level and 72 IMCI supervisors at district level on the Right Bank and 22 IMCI supervisors on the Left Bank. The supervisory system has covered with regular visits the IMCI-trained health providers initially with a first visit from outside team of evaluators/supervisors and then after local IMCI facilitators and coordinators took over the regular supervision function. The supervision is a well-coordinated process of regular visits, when the supervisor observes the practice of the FPs in following IMCI standards and producing quarterly reports. Informational support The job support for health providers included 9 item-packages for health personnel that included training modules and job aids, including patient assessment guideline and timers for counting breathing frequency. These were distributed to all health workers undergoing IMCI training. Trainers and 23

24 supervisors have also received additional technical support: guidelines for facilitators and for supervisors. The project has also covered the initial costs of design, printing and distribution of mother s agenda of over 200,000 copies and 85,000 copies of parents guide. The plan was for the Local health Authorities to take over printing and distribution costs starting with year Additionally, all district PHC centers were supplied with TV/video, projection screen and flipchart, paper for the flip-chart, for patient education and local training for health workers. Program expenditures The total amount of funds disbursed by UNICEF and its implementing partners for the IMCI program for the years was US$ 1,038,720, with most intense spending occurring in years (Figure 2) Figure 2 IMCI program costs, as registered by UNICEF Moldova, years $260,969 $19,645 $43,430 $12,223 $47,326 $130,919 $100,149 $152,110 $87,974 $85,783 $98,192 Yr 2000 Yr 2001 Yr 2002 Yr 2003 Yr 2004 Yr 2005 Yr 2006 Yr 2007 Yr 2008 Yr 2009 Yr 2010 No details were available regarding types of expenditures and other in-kind contributions in order to be able to analyze cost-effectiveness of the program. The in-kind contributions of the national counterparts are not possible to be assessed, as there was no monitoring in place of any IMCI-related expenditures, staff time and logistic support. Yet, the qualitative interviews revealed that key informants perceived the IMCI program to operate at a high cost-effectiveness rate and yielded important return of investments: All the financial inputs have been fully recovered, we have saved many lives of children, this is my personal opinion, I work with children on a daily basis and I see the effect key informant Perceptions of key informants and health managers regarding accomplishments and difficulties of IMCI implementation In-depth interviews with key informants and health managers depict a positive image regarding the IMCI program, its relevance and implementation in the Republic of Moldova. The program was deemed relevant by most interviewees, given the existing situation with prevailing preventable causes of death in children 0 to 5 years and is perceived to have had a significant impact on child mortality, as outlined in further on page 34. The success factors of the IMCI program in the Republic of Moldova were: Initial buy-in and support of Ministry of Health to the process of adaptation and coordination of the program Enthusiastic team of trainers, supervisors and facilitators Sufficient and sustained donor support and technical support for more than 10 years 24

25 Large coverage rate with training activities and informational support in the Republic of Moldova, IMCI was the first health program fully implemented in Transnistrian region Integration of a robust system of IMCI supervision and monitoring in the current health care system Integration of IMCI tool in the current health documentation Several contributing parallel processes have positively influenced IMCI program implementation: UNICEF s advocacy efforts to grant universal access to basic benefit package for children up to 5 years covered by the National Health Insurance Fund lead to enhancing the IMCI effect, and this makes Moldovan experience unique in the NIS region, according to one key informant Parallel behavior change campaigns aimed at changing caregiver knowledge and behaviors, e.g., Safe childhood, Safe motherhood, have probably also influenced a more active health-seeking behavior and alert level regarding danger signs in care givers. However, both key informants and health managers recognized the implementation was met with significant initial resistance of health providers and that the implementation phase has not achieved all its desired effects. Some underlying health system constraints lead to little health provider motivation to strictly comply with the IMCI standards: Shortage of PHC physicians and nurses that leads to work overload and does not allow sufficient time spent with patient or proper reporting according to the current standards. The time allocated to see a patient, 20 minutes still perceived to not being enough for a proper assessment, treatment and counseling and reporting The high turnover rate in physicians and especially nurses, with less effectiveness of CME of the incoming staff Resistance of PHC, especially those that have had a long work experience and have been retrained. Insufficient training coverage of nurses and not the same quality for the training (less than two weeks in medical colleges, less practical). A major constraint is the work experience of a health care worker. The younger physicians who have been initially trained on this program implement it easily, but those who have classic medicine training have harder time. A patient comes, he puts the stethoscopes and this is it. But based on IMCI it takes time to properly see a patient. Anything new requires time manager, Transnistrian region Key informants and health managers have noted some constraints of the implementation itself over time, and namely: Training coverage of nurses is lower than anticipated and the quality of training as integrated in the Medical College is lower than of the original training, as it is only an introduction and classbased. The rollout local trainings in raions for new incoming staff held by local facilitators are variable in quality and coverage. The hospital pediatric and emergency sectors have not been included in the IMCI training process and there is a discrepancy in practice between PHC sector and patients referred for hospitalization might be deterred from hospital admission. There is still no coverage with free basic benefit package in the Transnistrian region Drugs are not available in all PHC physician s office, and when mothers are referred to pharmacy, some pharmacist can advice on replacement one drug for another 25

26 Although acknowledged by everyone to be a tremendous help in informing and counseling mothers, reprinting of Mother s Agenda by means of local health authorities does not occur, only single positive experiences happening (e.g. Balti) and most PHC institutions face stock-outs in the current year. The IMCI program has not been revised since its initial adaptation and some things need revision, e.g., the list of IMCI drugs, and based on identified deficiencies through the current M&E system, to devise and implement short systematic refresher courses. Another underlying constraint as perceived by health managers is some established behaviors of caregivers so resilient to change, that information and counseling are not able to produce this change and need time and a generational change. And some structural factors have also been noted, such as education levels of parents, poverty, social vulnerability and migration. National IMCI M&E system The IMCI M&E system was set up at the national level through the Ministry of Health order no. 446 from December 3, The M&E framework includes designation of national and regional coordinating institutions, the process documents for data collection at the national level, regional and family physician levels, the list of process and outcome and impact indicators with their definitions and data collection. It also specifies that family physicians should fill in the IMCI patient evaluation forms for primary and follow-up patient visits on a daily basis, daily report the number of visits of children up to 5 years and make a monthly standard report. A local coordinator is responsible for quarterly reporting and yearly data collection based on an approved format. The national coordinator based at the Mother and Child Institute is responsible for validation and compiling the national report and submission by March 1 of the next year. Thus far, the national coordinator has issued three annual reports on IMCI program for the years Strengths of the M&E system 1. The national M&E system is a robust and well-designed data collection system, with a welldefined structure and periodicity of reporting. 2. All districts have local coordinators that report according to the M&E requirements on a quarterly and annual basis. 3. External evaluation by the local coordinators is provided on a quarterly and yearly basis Weaknesses of the M&E system 1. Although helpful at the set-up of the M&E system to discipline health workers, the high frequency of data collection and reporting is puts a significant burden on health providers and requires revisiting the periodicity of reporting (half or even yearly probably better than every quarter). 2. The analysis of some indicators has shown that they are difficult to be measured in practice and need to either be revised or eliminated from the current reporting requirements. a. Output indicators (e.g., percent of care givers who have been consulted for danger signs or percent of mothers counseled for advantages of breastfeeding) that require a quick survey of caregivers are unreliable and lack sufficient guidance as per data collection process 26

27 b. Some indicators have not shown sufficient reliability and usefulness in their collection, e.g., percent of hospitalized children who have been assessed through danger signs or sever sickness and need to be revised or excluded from the list. c. Overlap between process and output indicators lead to duplication in the same reporting form and needs to be revised and streamlined. 3. Our review has shown a variable quality of data collection and reporting by districts. Variability of interpretation and presentation of indicators in the district reporting forms does not allow for a standardized compilation of indicators at the national scale, as some districts report percentages only, without presenting the absolute numbers for nominators and denominators. In fact, only six districts (Chisinau city, Cahul, Cantemir, Cimislia, Calarasi, Telenesti) include absolute numbers of output indicators in their reports. Some districts, as mentioned in the annual reports provide unreliable data and probably need additional capacity building of their M&E capacities. 4. The annual reports note a discrepancy in the accuracy and reporting of some indicators when they are self-evaluated and when they are measured by external evaluators, putting the reliability of quantitative data under some indicators under question. 5. It is not clear how the weaknesses and recommendations identified in the annual reports translate into improved data collection practices. 6. The activities aimed at strengthening the quality of monitoring process and validation for data collection are not identified in the annual reports. It is not clear if districts with lower M&E quality have received any support in improving their M&E capacity. 7. It is not clear how the information provided in the annual reports is used for improving IMCI programming in raions. The observations from the qualitative interviews with health managers, local IMCI coordinators and health providers were that the M&E system is verticalized and is perceived to be formal and disconnected from decision-making process at the local level in improving raion IMCI programming. IMCI program outputs, national M&E system The national IMCI process indicators are likely to be more reliable than output indicators and they are presented by territories in absolute numbers. We had concerns regarding the interpretation and measurement of several indicators and we have excluded them from this analysis: the number of sick children of under 5 years, as most likely episodes of sickness are counted, not the number of actual sick children with several episodes in a reporting period the number of women who have been informed about danger signs, as the denominator is unclear and the absolute number is not linked to a cohort of children under 5 years The indicator percentage of children assessed with IMCI is deemed to be of little use by the local IMCI coordinators, but we have left it in the current analysis. Finally, in 2009 data for the city of Chisinau is missing from the national report for 2009, therefore limiting comparability of 2009 with the years 2008 and The process indicators show an increase in the percent of children of zero to five years assessed through IMCI tool from 66% in 2008 to 79% in The proportion of children that have been identified with 27

28 danger signs by health workers has been reported at 16% in 2008 to 8% in The percent of hospitalized children was reported at 33% in 2008 and 24% in The number of children diagnosed with dehydration who have received oral rehydration in the Oral Rehydration Corner (ORC) was 5% in 2008 and 3% in It is difficult to provide an interpretation to decreasing trends given the short observation period of three years and limited between-years data comparability, yet one hypothesis could be that the sickness episodes are less severe and do not lead to danger signs and hospitalization or a need for plan B rehydration. This hypothesis needs to be checked more in-depth. (Figure 3) The percent of children who received vitamin D prophylaxis at two years is at high levels of 95% in 2008 and 92% in Of the children assessed through IMCI tool, a third (31%) was diagnosed with anemia based on palm paleness in 2008 and the proportion decreased to 17% in The interpretation of this dynamic is unclear, since the incidence of anemia is at increasing levels, as presented in section on impact. Figure 3 Main national IMCI process indicators, year , Republic of Moldova, Right Bank only Percent of children assessed with IMCI Percent of children with identified danger signs Percentof hospitalized children Percent of children diagnosed with anemia based of palm paleness Percent of children with treated for dehydration in ORC Percent of children who received vitamin D at 2 years 16% 8% 33% 24% 31% 17% 5% 3% 66% 79% 95% 92% Yr 2008 Yr 2009 Yr 2010 Given the limited data quality of the district and annual reports, we have reviewed and were able to present only selected output-level indicators for six selected districts that have provided both nominators and denominators for their indicators. A national value of only one indicator (Right Bank only) is available, the percent of children assessed through IMCI, including 5 danger signs and 5 basic signs. It has registered an increase from 66% in 2008 to 79% in (Figures 4 and 5) 28

29 Figure 4 Percent of children assessed through IMCI tool, including 5 danger signs and 5 basic signs, selected districts and average, years % 100% 80% 60% 40% 20% 0% 65% 74% Figure 5 Percent of children assessed through IMCI tool, including 5 danger signs and 5 basic signs, total for the Right Bank, years % 75% 70% 74% 79% 79% Chişinău Cahul Cantemir Calarasi Cimislia Telenesti Average 65% 66% 60% 55% The proportion of children at two years who have received vitamin D prophylaxis according to the standard has been stable at an average 91% in 2008, 84% in 2009 and 90% in 2010, a discrepancy from the process indicator on Vitamin D prophylaxis presented above. (Figure 6) Figure 6 Percent of children under two years who received vitamin D prophylaxis according to the standard 120% 100% 80% 60% 40% 20% 0% 91% 90% 84% Chişinău Cahul Cantemir Calarasi Cimislia Telenesti Average 29

30 The percent of hospitalized children based on danger signs and severe sickness has seen a large variation, possibly because of difference in non-standardized data collection, with an average value of 32% in 2008 and 23% in (Figure 7) Figure 7 Percent of hospitalized children based on danger signs and severe sickness referred by PHC workers 60% 50% 40% 30% 20% 10% 0% 32% 19% % Chişinău Cahul Cantemir Calarasi Cimislia Telenesti Average The percent of children under two years who have received iron supplement for anemia prevention was reported at 30% in 2008, 24% in 2009 and 31% in 2010 (Figure 8), while the percent of children under five years who have received iron supplement for anemia prevention was lower, at 20% in 2010, pointing to still insufficient attention to detail for anemia prevention (Figure 9), although the process indicator above indicated a high occurrence of suspected cases of anemia based on palm paleness (17% in 2010). Figure 8 Percent of children under 2 years who have received iron supplement for anemia prevention 120% 100% 80% 60% 40% 20% 0% 30% 31% 24% Chişinău Cahul Cantemir Calarasi Cimislia Telenesti Average 30

31 Figure 9 Percent of children under 5 years who have received iron supplement for anemia prevention 100% 90% 80% Chişinău 70% Cahul 60% Cantemir 50% Calarasi 40% Cimislia 30% Telenesti 20% 19% 16% 20% Average 10% 0% Primary health care providers and IMCI Given the limited usefulness of the current M&E system in evaluating changes in levels of knowledge and behaviors of health providers and caregivers, this evaluation has included an ad-hoc survey and qualitative research of health providers and care givers. It has also helped to provide an additional data source for validating the results of the national IMCI reporting and program reports. The results are presented below. Socio-demographics A total of 127 health care workers have been interviewed for the purpose of this evaluation. Of them, 46.5% were from districts centers, 26.0% from rural areas and 27.6% were from urban areas (cities of Chisinau, Balti and Bender). On the Right Bank all health care workers were family medicine physicians, on the Left Bank they included pediatricians, internal medicine specialists and nurse practitioners. Most health care workers were females (92.9%) with an average work experience of 23.3 years (SD 10.5 years) and an average age of 47.8 years (SD 9.7 years). Inputs: Coverage with IMCI interventions The majority of interviewed family medicine physicians (90.6%) were covered with IMCI training. For most (96.5%) this was the standard 96 hour-long course and 3.5% mentioned shorter length between one hour and one week. Only 14.5% have received IMCI training in the past two years, while for 44.7% it was between two and five years ago and for 40.4% more than five years ago. Most providers who underwent training were mostly satisfied (60.5%) or very satisfied (28.9%) with the training they have received and only 10% were partially satisfied or dissatisfied with the training. As reasons for dissatisfaction were named short training length or too much time allocated to the IMCI training. The majority of physicians who were trained in IMCI have also received follow up supervision visit (95.6%). For 44.4% the first visit happened in the first 6 months after the training, while for the rest (46.3%) it happened later than six months after the training (9.3% did not remember exactly when the visit occurred). The majority (91.6%) found the first supervision visit useful. For the few of those who did not the supervision visit useful, the most often cited reason were that the visit was too stressful to be 31

32 instructive and they have not received enough feedback. The majority (66.7%) have mentioned that they have received more than one supervision visit and for most it is one time or less per year (14.9% less than once a year and 39.2% once a year), 33.8% mentioned once every half a year and 4.1% mentioned once every three months. Text box 1 Opinions of health providers about IMCI training The in-depth interviews with family physicians have confirmed a high level of satisfaction with quality and length of training and continuous supervision they have received and high coverage of health providers. "We are all of high opinion about the IMCI course we received in We received training materials. And the trainers not only presented lectures, but we had role plays, video materials. We had case studies. All our memory resources were put to use. We are very happy with it"' FP, Transnistrian region. "The supervisor observed how physicians interact with patients, then they checked the patient charts, she checked the immunizations room, temperature, rehydration supplies. They also talk to patients. After the supervision, they would also give some short lecture, e.g last time it was anemia". FP, city This was definitely not a superficial evaluation. The supervisor went on the hallway to ask mothers if they know about anemia prevention, if I gave them vitamin D." FP, town FP physicians that were formerly pediatricians and from the main cities were more skeptical about the training "The quality of training is good, but the content is too basic, it is intended for nurses", FP, capital city As for provision of training materials and job aids, 85.0% of physicians mentioned that they have the training modules, 94.5% have the patient evaluation booklet and 93.7% of them mentioned using it in their daily practice. Those who do not use them mentioned that they know already its content or do not have time, or other various reasons. The majority (83.5%) of family physicians mentioned using mother s card in their work with care givers and for those who did not use them the main reason was that they have ran out of copies. Some institutions managed to print lower quality black and white copies, others distributed photocopies and many simply stopped distributing them. 32

33 Text box 2 Opinions of health providers about Mother s Agenda PHC providers are all highly satisfied with the purpose, content and design of mother's agenda and use it extensively in their work with care-givers of children, physicians distribute them rather than an aid for counseling for feeding and development, while PHC nurse use them more interactively. The mother's agenda is excellent, I tell every mother: please read, I am not a teacher, but I will check and you should now everything", Pediatrician, Trasnistrian region "When I have a newborn at first home visit I take a copy of mothers agenda and I explain to the mother where to look, the danger signs, when to come see a doctor, when to ask for emergency care". Nurse, village The major difficulty with mother's agenda is frequent stock outs and interrupted supply of agenda. Many family medicine centers and offices do not have them at all. Several interviewees confirmed that they try to publish the agenda with their own means in poorer quality. We ran out of mother's agenda, we have ordered to print more from our PHC center budget, from national health insurance funds that we receive." manager, city The mother's agenda is great, before we had many, now I have only one copy. At some point we made photocopies and distributed them, but they look much better in color" FP, village Attitudes towards IMCI content When asked to prioritize the three most useful modules in their practice, physicians ranked the modules on assessment and management of a child with age between 2 months and 5 years (75.4%), on assessment and management of a child with age under 2 months (69.3%) and on treatment (48.2%), giving lower priority to child follow up, counseling of mothers and early child development modules. (Figure 10). Figure 10 Opinions of FPs about the most useful IMCI modules Text box 3 Strengths of IMCI 33

34 PHC providers mentioned as strengths of the IMCI training content to their practice the accent on prevention and communication, empowering mothers to be responsible for their child's health; being better equipped to evaluate fast the child status and refer to the appropriate level of care (home, outpatient or hospital); providing clear framework for clinical actions. The IMCI approach disciplines me and does not allow missing anything; you have to count breathing frequency, measure fever etc. Physician, village "Our patients say that they like fewer medicines and more attention" PHC physician, Transnistrian region "Before the physician was responsible for child's health, this strategy makes mother empowered to be responsible for her child's well-being, of course, the physician participates and educates her'. Physician, city Relevance: Attitudes of health care workers towards IMCI program implementation Interviewed physicians were asked to assess their level of satisfaction with implementation of IMCI strategy in their facility. Almost half of interviewed physicians were mostly satisfied (48.8%) and 10.2% of them were very satisfied with the IMCI strategy implementation, while over a third (36.2%) were only partially satisfied and 3.9% were dissatisfied with this strategy. (Figure 11) Figure 11 Level of satisfaction with the quality of IMCI implementation in the health institution Text box 4 IMCI relevance to Moldovan PHC practice 34

35 The relevance of IMCI is perceived quite differently by health providers. Some have embraced the IMCI strategy as a great help for them, especially nurses, PHC physicians that were initially trained as general medicine specialists and those with shorter work experience and those based in district centers and rural areas. "IMCI is good for us, I was a internal medicine specialist, so IMCI is very helpful for me" PHC physician, village "I had a case of meningitis that the nurse has identified after filling in the IMCI evaluation form. She said she would not have suspected it unless she filled in the form. Another case was at a home visit, where the child had intercostals withdrawal and the nurse called the ambulance. My nurse is really good with using IMCI, although I can't work according to IMCI every day" Physician, district center "IMCI works great for children who are not frequently sick" PHC physician, village There were quite a few IMCI skeptics among the providers, especially PHC nearing the retirement age, former pediatricians and those living in Chisinau, Balti and some district centers. They consider the IMCI approach more relevant to less developed countries and not for physicians; the IMCI patient evaluation is perceived to be in conflict with the national standards for medical practice in diagnosing and prescribing treatment and an extra burden with little clinical use. "IMCI is good, but it is designed for Africa, they showed us video materials from Africa, I can't put a diagnosis looking in his palm [for anemia]. I need to refer to a blood analysis for Hemoglobin count, we are not Africa". Physician, city "We implement IMCI because we are required, I do not see any use. I have a long practice and I have treated children without antibiotics before, too." Physician, district center Sometimes children do not fit the IMCI evaluation algorithm, their status being much worse then what the evaluation shows us. He does not have fever, no frequent breathing, yet he has severe pneumonia. Physician, village Physicians were asked to pick maximum three difficulties from a list read to them. By far, the most often mentioned difficulties were regarding filling in assessment forms in child s card, namely that it duplicated some other paperwork (55.6%), that it was not relevant for their practice (8.7%) and it takes too long (7.9%). Additionally, over a third (36.5%) mentioned that it takes too much time to assess a child and do proper follow according to IMCI guidelines, in conflict with other responsibilities (9.5%). Only 10.4% of PHC physicians mentioned there were no difficulties to implement IMCI strategy in their institution. Over a third of physicians (36.5%) considered that IMCI should better target and be implemented by nurses rather than physicians. (Figure 12) Low shares linked difficulties of IMCI implementation with insufficient coverage with training (18.3%) and 11.9% considered that the fact that not all nurses have been trained was difficult for IMCI implementation. At the same time, very few (7.9%) mentioned IMCI not being relevant for the country as one of the top difficulties. Figure 12 Perceptions of health providers about difficulties and barriers in IMCI implementation 35

36 IMCI difficulties as perceived by family physicians Both survey and structured interviews revolved a lot around complaints of health providers about several key difficulties in implementing IMCI strategy. Most difficulties had to do with the time it took to fill in IMCI patient assessment forms and the M&E forms, but also with the restrictions imposed over antibiotic use and the low acceptability of some drugs included in the basic benefit package. Text box 5 Difficulties in IMCI reporting, as perceived by health providers As most complaints from service providers were related to filling in patient evaluation forms and reporting, we looked into it more in-depth in qualitative interviews. PHC providers have a huge workload given insufficiency in staffing, some physicians work on several shifts, some see up to 50 patients a day. They think the time allotted to seeing a child of 15 or 20 minutes is insufficient for proper examination, prescribing, counseling, and filling in the IMCI form. We are overloaded with reporting. We have many patients and we need to write in seven different places and make eight reports for the same patient. Patients are as many as they are, but we would definitely need less unnecessary paperwork. Nurse, village "Today I saw just one child, I filled in his form, but when there are many you just are not able to keep up the pace. The evaluator would say that I saw 30 children, but filled in forms only for 24. I just do not have the time" PHC physician, district center "To establish the number of breaths takes time. If the child sleeps it is easier, if it is active it is impossible, so I ask the mother to count while she waits for me and report." PHC physician, village Most frustrations are also caused by the fact that physicians do not find useful this paperwork, because of discrepancy between the national standards, DCM classification and duplicates their time to fill in. "Honestly, I do no find it useful in establishing a diagnosis, it is just extra paperwork." Physician, city 36

37 If IMCI and the national standards were the same, it would be easier. We see all the IMCI signs, but I need to add auscultation, objective status and this is not part of the IMCI evaluation form. How can I write: possible infection? I have to write a DCM diagnosis, Acute respiratory infection or influenza. If it is pneumonia, according to the standard I need to refer the patient to X-ray and not only by number of breaths". Physician, city If I only follow IMCI, I cannot know what happened to the child three days ago. The breathing frequency does not always mean he has respiratory sickness; I need to check with stethoscope. this child. Physician, district center "E.g., for a child with a cold, cough is included, but what I hear in his lungs is not included. Then I write a diagnosis, acute bronchitis based on what I heard in the lungs, but there is no space to write it in the form. In case of a child who is often sick we run out of space so quickly that we need to add additional papers with glue and these patients charts become too thick and unhelpful." Physician, district center "For immunizations, I do not need to write down in the IMCI form every visit, because we have a separate form to monitor it". Physician, city The time to fill in a patient evaluation form varies between one and 15 minutes, depending on the level of acceptance of IMCI forms by the physician, the experience of filling in this form, caseload and patient disease and frequency of visits. Some physicians are so behind with filling in forms that they take them home in the evenings and week-ends especially before evaluation visits. Many physicians questioned its usefulness to establish a diagnosis, given the difference in classification according to the national standards and IMCI. "I fill them in, when I finish them; I either make copies or write on white paper. If the reason to come see a physician is runny nose, I do not waste a form for this. Sometimes I write first visit and follow up visit on the same form." Physician, village Nevertheless, there were several physicians who got used to using them and did not find them burdensome: "It is easy to fill in. You just check things and save time. But I still need to write down on the other side, so it duplicates the work." Physician, village "I got used to this evaluation form. For many years, I go to Chisinau to make 500 copies, because it is cheaper and I add them to the charts. The administration says they have no resources, so I am paying for this myself. I need to be reassured that I did everything properly". Physician, village The common suggestion was to better integrate the national standard, DCM classification for diagnosis with the IMCI form. The children charts should include objective examination, too and leave space for the diary part. Physician, district center Another often mentioned reason for discontent with IMCI is the choice of medicines covered under basic package, especially antibiotics "All the drugs covered by NHIF are bitter" Physician, village 37

38 'Children do not tolerate well Hemofer [iron supplement for anemia prevention and treatment] that is covered by NHIF. We are imposed to prescribe something else that parents need to pay out-of-pocket. Physician, village "I think IMCI was designed many years ago and things have changed, some medicines need to be revised, especially to revise 1 st line and 2 nd line medicines" PHC physician, village The interviews revealed an extensive resistance of physician towards IMCI strategy of limiting prescription of antibiotics. Physicians wish they had more flexibility in the choice of antibiotic and find creative ways to overcome the rigid system. Some recognize a long established practice of overprescribing antibiotics in cases when they are not necessary in children. "The difference between a physician who works according to IMCI and another one who does not is prescribing treatment: the first would only prescribe 1 st line or 2 nd line antibiotic, while the second will add a whole list of medicines." Nurse, village "I wish to have better antibiotics covered by NHIF. We only prescribe Amoxicillin, the same every month. And when a mother comes again to ask for Amoxicillin, when it is not required I am ashamed. When mothers do not want to go to a hospital, we are imposed to prescribe antibiotics." Physician, village "I know Amoxicillin will not help in this case, as we have the sensitivity test showing resistance to Amoxicillin and I have a hard time prescribing the right antibiotic." Physician, district center "According to IMCI you cannot prescribe antibiotics to the right and to the left. Everyone fears complications of respiratory infections and some colleagues consider that antibiotics are their only method to prevent complications of viral infections. Not all mothers come after 2 days for a follow up visit and after 3-4 days the status of the child can worsen, they call an ambulance and get hospitalized and it shows we have not done our job." Physician, village "In order to prescribe an antibiotic, we sometimes write a different diagnosis, e.g. ear infection, or we write something according to IMCI then prescribe another medicine for treatment." Physician, city Outputs: knowledge and practices of health providers Knowledge of major danger signs Most family medicine physicians (89.8%) were able to name three or four signs out of four listed in the questionnaire (62.2% mentioned all four signs and 27.6% mentioned three signs). Only one person could not name any signs, 1.6% mentioned only one major sign and 7.9% mentioned two signs out of four listed. At the same time, some 58.3% named other symptoms as danger signs and the number of other mentions ranged from one to seven other signs for which medical attention is needed immediately, showing an increased alert towards child health by primary care physicians. (Figure 13) Figure 13 Knowledge of danger signs, family doctors 38

39 Knowledge of pneumonia management Primary physicians are well aware of IMCI-specific signs for diagnosing pneumonia as most (93.7%) have mentioned frequency of breathing and intercostals retraction (83.5%) as pneumonia specific signs, while less than half (43.3%) mentioned coughing. At the same time, the majority also mentioned other signs, especially auscultation signs as important for establishing pneumonia in children. Figure 14 Knowledge of IMCI-specific signs of pneumonia, family doctors Although most physicians mentioned breathing frequency as a major sign to diagnose pneumonia, only 71% were able to correctly identify the normal frequency of breathing in a two-year child of 40 breathings per minute. As for treatment, almost all physicians (94.5%) correctly mentioned Amoxicillin as the antibiotic of choice in treating pneumonia in children. 39

40 Management of diarrhea Most physicians (95.3%) were able to name at least three signs and symptoms of diarrhea they assess in a child according to the IMCI listed eight signs. The objective signs were mentioned more frequently compared to the questions asked to care givers. The most often mentioned signs were dehydration signs: skin goes back slowly, sunken eyes and how the child drinks water. Figure 15 Knowledge of IMCI signs of diarrhea, family doctors As for diarrhea management, only 66.9% of care providers were able to correctly identify the correct management tactic in a child with a moderate diarrhea as plan B, outpatient oral rehydration under physician supervision. Management of anemia Nearly everyone (99.2%) mentioned the IMCI specific sign for anemia assessment based on palm paleness, while 62.7% mentioned additional one to six other signs necessary for anemia assessment, including lab test for hemoglobin. Early stimulation for development Over half (60.6%) physicians were able to name three or more early stimulation techniques (out of listed in Mother s agenda of six stimulation techniques in a child with age of 6 to 12 months). The less frequently named stimulation practices were caressing a child (18.1%), holding a child as often as possible (18.9%) and reading to an infant (37.0%). Some 34.6% mentioned other stimulation signs not listed in the mother s card and included massage, early development of self-feeding skills and potty training as early stimulation methods. 40

41 Figure 16 Knowledge of IMCI advice for child development at 6-12 months, family doctors Text box 6 Counseling for child feeding and early stimulation Physicians admit that they do not spend enough time for counseling and are limited because of the patient load and short time allocated for seeing a patient. Most of them focus on counseling during first home visit and then only reinforce advice at follow up visits. Most emphasis is made on feeding counseling and encouragement of breastfeeding and this is definitely seen as a positive outcome in quantitative data on breastfeeding rates and positive trends in delaying introducing solids as well. Yet, physicians still encounter discouraged practices, such as feeding cow milk to babies under 1 year old. "Well, you ask her if she understood everything and she says yes, but when you ask in more detail you see she did not get it, so you start again. We have started spending more time on counseling and we see they indeed need this." Physician, village "I spend most of the time to counsel a mother when I make home visit. Then it might take up to an hour and a half to discuss with her about feeding and stimulation practices." Nurse, Trasnistrian region "Only when I go to a home visit I can spend more time for counseling. I have time to observe the living conditions, breastfeeding or feeding. There are still mothers who feed their children with diluted cow's milk. Yesterday I saw a one month-old baby and her mother gave her cow's milk with cereal. It is rare, but how can this happen if we tell them every time that no cow milk is allowed? And unfortunately I know the answer, they try to find an inexpensive way to feed their babies, even knowing about the risks to their children, not everyone can afford formula at 100 lei per week." Physician, village "To be honest, I do not have time. We talk, but not as long as we should. After 35 patients or more a day what can we do The emphasis is put on sick persons, for prevention there is little time." Nurse, village "I would say 10-20% ignore medical advice and listen more to their older relatives." Physician, village As for counseling for early stimulation, this is the most overlooked area in the IMCI strategy and physicians admit to not pay attention to this section. Some feel they do not know sufficiently well and mothers now have better sources of information. 41

42 "We do this [counseling for development] less. We mostly make sure the child has a good physical development and we pay attention to speech. We ask this every month." Physician, village "I do not speak about child development with every mother. Now mothers read themselves on Internet and they know more than we do. Only if I see the child is behind his age, then I advise the mother to spend more time with her child, play more, to talk more." Physician, village Perceptions of health providers about outcomes and impact Many physicians see the effect of IMCI approach first in how they have changed their practices towards more counseling, more patient education and more evidence-based prescribing, and second changes in behaviors of caregivers, especially in feeding practices and behavior when they notice danger signs. Some physicians have also noticed impact-level changes, e.g. fewer severe sickness cases and fewer mortality cases. Text box 7 Perceptions about changed behaviors and practices Many physicians talk about their changed attitudes and practices after implementing IMCI approach and they make a direct causal link to improved knowledge and health-seeking behaviors of care givers. "Mothers are more responsible for their children. After the IMCI course I have started talking more with mothers about everything, give them the agenda and mothers are satisfied." Nurse, village "As a pediatrician, I have changed my treatment strategy, if a mother comes to see me and I give her the appropriate attention, many times I avoid prescribing antibiotics." Physician, village "We had a case when a mother knew well the danger signs and she saw that the child vomits after every meal, she took the child to the hospital and he was diagnosed with meningitis. And this is because she knew the danger signs. Of course we know all of them, but before we did not consider mothers should know them, too." Physician, district center "Of course I see changes. We start now introducing solids at six months and I see fewer cases of allergy and disbacteriosis and more breastfeeding" nurse, Transnistrian region "Mothers now count breathing frequency and diarrhea by themselves before coming to see a doctor." Physician, village "The most difficult to change is the responsibility of parents: I gave birth of the child, you should take care of it.[...] They act as if they gave birth for me." Physician, Trasnistrian region Perception about Impact "We do not see so many emergency cases. We have IMCI for five years now, before we had mortality cases, now they are really rare and they happen not because of doctors, but because of parents who neglect their children or because children are left with grandparents and they do not care well for their grandchildren". Physician, village "Yes, things have changed. I worked as a pediatrician in hospital and I remember seeing children with severe dehydration, one has died, I still remember that case. Now it does not happen at all". Physician, village 42

43 Coverage with key IMCI interventions, knowledge and practices of care givers of children age 0 to 5 years Sample demographics The sample consisted of 190 care givers sampled from the health-seeking care population of care givers of children aged 0-5 years. The sample covered 32.1% of care givers residing in the cities of Chisinau, Balti and Bender, 44.2% of them in district centers and 23.7% of care givers from the rural environment. The majority of main care givers were mothers (94.2%), followed by grandmothers (5.3%) and fathers (0.5%). The mean age of mothers was 28.2 years (SD 5.9 years) and of grandmothers 52.6 years (SD 5.8 years). As for the education level of mothers, they had medium level (43%), university level (31.1%) and postgraduate studies (2.6%), while 5.8% have graduated from high school and 15.3% of mothers had general school level. Child feeding practices The vast majority of children (95.2%) were breastfed since birth. In the subsample of those who already finished breastfeeding their children (n=124), over a third (37.1%) were breastfed for six months or less, about a third (26.6%) were breastfed between six and 12 months and over a third (36.3%) were breastfed over 12 months, while 3.1% of women have breastfed their children after 24 months. (Figure 17) The average length of breastfeeding was 10.7 months (SD 7 months). In the subsample of caretakers of children of 6 months or less (n=32), 66.6% of babies were exclusively breastfed in the past 24 hours prior to survey, while 24.2% were given vitamins and minerals, 21.2% were given water and/or juices and tea, 18.2% of babies were given formula and 3% were given cow milk. Figure 17 Length of breastfeeding, health-seeking caregivers, excluding current breastfed babies Compared to ECD 2003 and 2009 findings, breastfeeding over 12 months was on the decrease in this sample (36.3% versus 43.1% in ECD 2009 and 49.9% ECD 2003). 43

44 Knowledge about optimal length of exclusive breastfeeding was rather good, as 66.7% correctly mentioned 6 months and additional 18.5% mentioned ages between 7 and 12 months, while only 2.1% mentioned ages less than 3 months and 9.5% mentioned ages of 3-5 months. The average age of introducing solid food was 5.5 months (SD 1.7 months), a sign of appropriate timing of introduction of solids. Half of the caretakers in the subsample of children 12 months or older (n=156) introduced solids at 6 months and an additional 13.4% introduced solids sometimes between 7 and 12 months and only 8.3% started giving other foods before 4 months. Figure 18 Timing of introducing solids, health-seeking caregivers, subsample of caregivers of children 12 months -5 years before 4 months months at 6 months 7-12 months Compared to ECD 2009 survey, it appears that this sample of caretakers accessing care have better practices in delaying introduction of solid food after six months (35.7% versus 70.2% in ECD 2009 survey introduced solid food before 6 months). Knowledge about correct order of solids between 6 and 12 months was also good, as most mentioned vegetables (88.9%), cereals (77.9%), fruits (76.3%) and meat (71.1%), with a more reserved attitude towards dairy (54.7%) and egg yolk (51.1%). Text box 8 Child feeding in first year of life Interviewed mothers put a lot of attention to feeding issue and this is one of the best discussed and known areas. Mothers are aware about the benefits of breastfeeding and the need to introduce solids later, after 6 months and avoiding cow's milk. Yet there are exceptions, under economic pressure or the influence of grandmothers. "The baby will be healthier if breastfed, if solids are introduced too early, then the baby has problems." caregiver of a 6-month-old baby, village "I know I can't give cow's milk until de child is one year old, cow's milk is good for adults, but not for babies" caregiver of 8-month-old baby, district center 44

45 "I had little breast milk and I gave her NAN (formula) for about 4 months and then I did not have enough money, so I started giving cow's milk at about 6 months. Our parents raised us the same." Caregiver, 1 year and 5 months, village "I started introducing mashed vegetables at 4 months, after 6 months I started giving diluted cow's milk and I make porridge with milk. [ ] I did not talk to doctors about this, I know myself, since this is my third child", caregiver of an 8-month old child, district center In the subsample of caretakers of children with age between 6 and 12 months (n=33), most care givers have already introduced and provide on a daily basis cereals, vegetables and fruits, a half have given meat (51.5%) and very few mentioned fish (9.1%) as food provided in the past 24 hours prior to survey. Figure 19 Distribution of major nutrients in the 24 hours preceding survey, subsample of caregivers of babies aged 6-12 months In the subsample of caregivers with children with age between 12 months and 5 years (n=124), most caretakers mentioned the main groups of products (Figure 19), yet only 60.7% of them have fed their children with all four important groups (meat or fish, fresh vegetables and fruits, dairy products and grain products) in the past 24 hours prior to survey, pointing to feeding deficiencies of daily provision of the important dietary products. 45

46 Figure 20 Distribution of major nutrients in the 24 hours preceding survey, subsample of caregivers of babies aged over 12 months to 5 years Text box 9 Feeding practices in children over one year Daily intake of dairy, meat and fruits is more dependent on the income level than level of knowledge of caregivers, the rural population being the most disadvantaged. Most parents try their best to provide a wide range of recommended products and some recognize they themselves do not eat them, but provide them to children. Meat and fish Most parents try to feed their children meat every day or as often as possible, but fish is available more rarely, once a week to once a month. "I give them meat 2-3 times a week, fish once a week, depending how often their mother sends us money". Caregiver of a 2-year old and a 4-year old, village "I give my child meat almost every day. We do not buy, we use our own poultry". Caregiver of a child of 1 year and 5 months, village "I put liver in mashed potatoes, but I have not given meat yet. He is allergic to eggs and tomatoes". Caregiver of 1 year and 4 months, village "She likes fish. Once a month we buy fresh fish, so that fish is still is alive and I know it is fresh. Caregiver of child, 3 years, village "For me healthy eating means fresh and diversified food, and not a lot. Every product has its advantages, but there are some that we cannot afford. I know fish has calcium and good fat, but it is too expensive". Caregiver of child, 3 years, district center Dairy products Intake of dairy products depends if the household owns cattle or they buy dairy products. Some mothers state their children have dairy products every day, either drink milk or they add to other foods, or they 46

47 eat cheese. Most rely on buying directly from farmers, and very few mentioned using bought dairy products, such as yogurts. "We have our own cow, so we make all the porridge with cow milk, then macaroni with milk". Caregiver of 1-year-old child, district center "I give milk every day we buy milk only from one source; if I take 3 liters it is enough for 3 days". Caregiver of a 4-year old and 2-year old child, village Fruits and vegetables The most frequent fruits are apples, and provision of other fruits depends on financial status. From vegetables, fresh carrots and beats are most used. Frequency of fruit intake is dependent on family income and season. "During the winter not so many fruits. We mostly rely on apples. There is a week when he eats every day 1-2 apples and another week when he would have only 1-2 apples that week. When we have money, we buy bananas, oranges". Caregiver of a 3-year old child, village "We buy fruits, apples once or twice a week; sometimes I would buy banana, one banana a week. In the summer I would buy strawberries and raspberries two times a month, we do not have our own". Grandmother of a 2-year old, village Vitamin D A high share of care givers (95.3%) provided their children with Vitamin D and in correct frequency (68.1% mentioned administering Vitamin D every day and 24.7% 3-6 times a week). The vast majority (94.3%) has started providing Vitamin D within the first three months (68.2% within the first days or month of life) and the average length of Vitamin D intake was 14.6 months (SD 7.7 months) with a range from 1 to 36 months for those who finished providing Vitamin D at the time of interview (n=107). Text box 10 Knowledge about anemia prevention in care givers Qualitative interviews have shown an increased awareness of mothers about anemia prevention and some provided more correct answers than others about what are the iron-rich dietary sources. "To prevent anemia, you have to give apples, beet, red meat but also can supplement with ironcontaining medicines" caregiver of a child, 1year and 7 months, Transnistrian region "The child should eat a lot of vegetables, fruits, meat and fish, to prevent anemia" caregiver of a 3-yearold child, village "We need to provide iron-rich diet, but I also asked my doctor to give me iron supplement, because when I gave birth I had anemia" caregiver of a child, 2 and a half years, Transnistrian region "I know liver is good for anemia [...] and one nurse told me cheese with sugar is good for increasing hemoglobin. The medicine Hemofer works, but he had stool problems, so I had to buy "Hemoglobin plus" and Immunal to increase immunity", mother of a frequently sick child of 1 year and 4 months who does not give him meat or eggs yet, village 47

48 Immunizations The majority of care givers (86.3%) stated they have received all immunizations according to the immunization calendar and for those who did not receive them in time (26.8%), the major reason was delay due to sickness (67.9%), while 14.6% mentioned being against immunizations and 10.7% do not believe in immunizations, indicative of an active negative opposition to vaccines for every fourth caretaker who did not vaccinate their child. The majority (71.1%) mentioned to have a yellow immunization card, and some territories were systematically not providing immunization cards (e.g. Cahul, Ciadir Lunga and Bender). Text box 11 Opinions of care givers on immunizations All the interviewed caregivers took their children to immunizations, yet their opinions about them are quite varied. Some immunize their children because they believe in benefits, while others have mixed feelings about immunizations and make them because of pressure from doctors. A narrative of bad consequences of vaccines for children health is present and the opposition is stronger in cities compared to rural areas. "You need to make all immunizations, now there are all these diseases, e.g. flu, and the child is best protected if he has all immunizations made in time." village "Immunizations are good, this is how I was taught home, we were immunized when we were little and now I take my children to all immunizations". Village "We do not have all the vaccines and the doctor insists we make all of them. To be honest, I want to make the immunizations in Chisinau, because I am afraid to make them here. I made only one here that I knew was not that risky" Transnistrian region "I have doubts if immunizations are a good thing, they talk about the bad consequences on TV, then there are complications and children have died, at the same time doctors say they are needed", village "There are cases when vaccines lead to paralysis, this is hearsay, from doctors and nurses I know, they think immunizations are not good these days, and there are many experiments on people" village Opinion of health providers on barriers to immunizations The reticence of caregivers towards immunizations is corroborated by perceptions of health providers as well. They think that health providers need to spend a lot of time on convincing, to be able to reach the immunization targets. Yet, a certain share of caregivers are so strongly opposed to immunizations, that nothing will convince them to immunize their children. The strongest opponents are caregivers that are from religious factions. "We have eight families in our village who have not received a single vaccine, parents are categorically against. The majority are from religious sects. People from preventive medicine talked to them, the chief physician talked to them, but they have signed a paper that they do not allow their children to be immunized." Nurse, village "I have two religious mothers with many children who do not come for antenatal follow-up and refuse any medical care. They do not immunize their children, I went to talk to them, the social worker did, the 48

49 nurse did, and even the policeman and we could not dissuade them. I think only 10% could be convinced, they are so confident that vaccines are bad for health, they do not take Vitamin D, iron supplement for anemia either, nothing". Physician, village "I think we need a national behavior change campaign to promote immunizations." Physician, village Knowledge of danger signs A total 88.4% of care givers were able to list at least two danger signs in a child or other situations when caregivers need to urgently see a physician. The most often known signs were fever over 38 0 C (90.5%), child vomits everything (48.9%), and worsening condition (46.3%) and child is unable to drink or eat (44.7%). Figure 21 Knowledge of danger signs and situations needing urgent medical care, health seeking caregivers As for the number of danger signs, some 27.9% could name 1-2 signs, 48.9% know 3-4 danger signs, 21.1% 5-7 danger signs, and 2.1% could not name any. In comparison with ECD 2009 survey findings, the knowledge of danger signs in the sample of caretakers accessing health care is higher (88.4% compared to 81.0% in ECD 2009, 82.9% in ECD 2003 and 73.0% in 2000 named at least two danger signs). Text box 12 Knowledge of danger signs by care givers The overall perception reinforced by the qualitative interviews was a heightened alert in any symptom as a reason to see a physician for both mothers and doctors. Many caregivers confuse reasons for seeing a physician with danger signs. "We have the danger signs written, at the hospital they wrote them and gave them to me as well as phone numbers where to call", village "I know danger signs. when there is high fever over 38 o C, at 37 o C, you can stay home but not at 38 o C; when the child coughs; when he has a skin rash; whenever you see anything wrong you must go and see the doctor", village 49

50 "I need to go see a doctor in case of high fever over 38C, diarrhea, when the child does not eat, but if I see anything wrong, I go immediately, because I am afraid of anything", village Diarrheea The majority of caregivers (70.5%) were aware they need to provide more liquids than usual to a child with diarrhea, however there were still those who thought their child should drink less (14.2%) or not at all (2.1%). Basic IMCI knowledge indicators in caregivers To summarize the key indicators measured in this sample of caregivers that access health services compared to the results of the total population as measured by a household survey, we notice better indicators in the health-seeking population in the proportion of caregivers introducing solids at 6 months (63.7% in this sample compared to 29.8% in ECD 2009) and increased knowledge of at least two danger signs (88.4% versus 81.0%). The interpretation could be the effect of exposure to more patientphysician interaction and effect of IMCI strategy. The limitation is the limited comparability of household survey findings with a small sample of this survey. Table 3 Comparison of basic indicators between caregivers accessing health services and household ECD 2009 survey Indicator MICS 2000 ECD 2004 ECD , Caregivers accessing health services Breastfeeding over 12 months N/A Introducing solid food at 6 N//A months or more Knowledge of at least two danger signs Physician Counseling The vast majority of caregivers (90.0%) stated to have talked to the physician about the benefits of breastfeeding, 82.1% about introduction of solids and 70.0% about diet of a sick child. In addition most mentioned talking mostly about child feeding (82.1%), immunizations (85.3%) and danger signs (77.4%), while the topic of child development was the least covered (56.8%). Figure 22 Physician counseling on IMCI topics, health-seeking caregivers 50

51 Text box 13 Physician focus on counseling, as perceived by caregivers Feeding counseling The qualitative interviews have confirmed the heavy emphasis of physician on feeding counseling, although many interviewed mothers say they read themselves on Internet or talk to their mothers. "Our family physician told us to give meat and fish daily, it is good for children", village "It is easy to discuss with the physician, he is friends with my mother, but I have never asked him about feeding, my mother is my best advisor, she was the one to raise my brother's child an my older daughter", village "I can ask anything my family physician. The other day I told her I breastfeed at only one breast and she asked if anything bother, she checked my breasts; she asked if I have fever, she measured it. She told me to put the baby more at the second breast, she reassured me it will start making more milk", village Early development stimulation counseling The situation with counseling for development is quite different and physician input is less frequent and sustained than for feeding. Many mothers put themselves emphasis on early stimulation, regardless of the counseling they received. Some mothers thought stimulation for development mean giving vitamins, massage and exercises for their children, the outdated approach to child development. More mothers from villages have fewer books and admit reading less to their children. "/ play with my child, we have many development toys, we do everything together, we read books, all day long we are together", Transnistrian region "I have a book from my older son and starting the age of one year I show him the animals and the sounds they make " village "The doctor showed how to correctly make massage to my boy. He also told me that I need to talk to my child. Until the age of 1 year the doctor talked to me more, but when he she saw that we do well, she spends less time on us", Transnistrian region 51

52 "What stimulates my child? First of all vitamins; if the child has vitamins, this strengthens his bones and brain, but it also depends on how parents educate their child, village "I do not have time to read to him, he sits next to me and we learn poems by heart, he tells one after another. He has an older sister and she read to him, I don't. [ ] Yes, the doctor told us how to massage, how to walk the child",village "Well, the doctor mostly checks how my child develops, he does not give much advice. They only give advice at first child", district center "I do not remember the physician talking to me about development, I usually go to see my family doctor when my child is sick, I mostly discuss health issues", village "The physician came home a couple of times, he told that that I raised other children and I should know what to do", village Mother s agenda Over two thirds of care givers (71.6%) have seen a mother s card and of them, 99.3% were given mother s card by a physician, 89% still have it and 88.6% have read it. A half (51.2%) thought all information in mother s card was useful and the most often mentioned sections were child feeding (14.0%), alert signs (10.1%) and development (7.8%). Text box 14 Opinions of caregivers about Mother s Agenda Some caregivers have seen mother's agenda and have them at home and gave a positive feedback, while others have not seen it or thought it had insufficient information. "It has lots of information in it, if something happens, it is written what to do, just read and do it" village "I have one of these at home, I read it. It is not good for every child; my child for example does not like buckwheat", Transnistrian region "We have been given mother's agenda and some green box that we should have by the window and learn everything by heart, e.g., at what height you should keep the medicines, because someone were to come with an evaluation, but no one came", village "Yes, the physician gave it to me, but this is only a leaflet. I know there are books that have more information and I keep asking the physician when they are going to be available", village Access to medical services Almost two thirds (64.6%) sought a physician for a sickness episode of their children in the six months prior to survey. Of them, 30.1% ranked this last episode as minor, 48.0% as medium and 22.0% as a severe sickness and almost everyone (96.7%) went to see a doctor. The primary point of access for the majority was the family medicine physician (67.8%), while for 15.7% it was the pediatrician or a physician of choice (4.1%) and an additional 7.4% went directly to hospital and 5.0% to emergency room. Nearly everyone (95.9%) was prescribed medicines, but only in 25.4% of cases health insurance covered completely the cost of drugs and for over a third (34.7%) health insurance covered only partially the 52

53 costs of prescribed medicines. Geographically, on Left Bank in Bender and Grigoriopol more than 80% of caretakers had to pay entirely for prescribed medicines and in Chisinau and Balti about half paid themselves entirely for medicines, while in raions people rely more on health insurance to cover the costs of medicines. Figure 23 Degree of coverage of cost of drugs by health insurance Satisfaction with primary health services The majority of caregivers thought the services they receive in family medicine centers were average (39.5%) or good (37.9%), while 16.8% thought they were excellent and 5.3% thought they were poor, showing a rather good level of patient satisfaction. As for what needs to improve, 38.0% mentioned better communication with physicians, 34.9% better management of services and no waiting time and lines, 30.7% access to a wider range of compensated medicines and only 4.8% mentioned informal payments, while 37.4% mentioned various other reasons. Text box 15 Opinions of caregivers about their interaction with family physicians The opinions varied widely among caregivers from excellent to very bad and depended mostly on the personality of the doctor. Even when the physican-patient interaction was excellent or good, a common complaint was long waiting times and short time spent with physician. "It depends. Some doctors explain everything and others who do not care if you understand or not, some are badmouthed, others counsel you and explain what one should do" village "The doctor explains everything, she goes into detail to explain how to take every medicine, after two days I come for follow-up and then we decide together if to continue, decrease, etc. I have no complaints, I can call at any hour to ask anything" village "I speak comfortably with my doctor, but he has little time, there are many patients and very few doctors, so I write down all my questions on a paper to ask them all and not to forget anything because it is all in a rush." Transnistrian region 53