Lady Health Worker Programme

Lady Health Worker Programme External Evaluation of the National Programme for Family Planning and Primary Health Care Training Programme Review Oxford Policy Management March 2002

Reports from this Evaluation 1. Final Report 2. Quantitative Survey Report 3. Financial and Economic Analysis 4. Punjab and ICT Survey Report 5. Sindh Survey Report 6. NWFP and FATA Survey Report 7. Balochistan Survey Report 8. AJK and FANA Survey Report White Cover Reports 9. Training Programme Review 10. Qualitative Report 11. Survey Sampling Design 12. Quantitative Survey Questionnaires

ACKNOWLEDGEMENTS This evaluation of the National Programme for Family Planning and Primary Health Care was undertaken by Oxford Policy Management, UK and funded by the Department for International Development, UK. The core evaluation team included, in alphabetical order: Shafique Arif (Survey Manager); Dr. Abdul Ghaffar (Systems Analysis & Dissemination); David Hoole (Financial & Expenditure Study); Simon Hunt (Team Leader); Prof. Fehmida Jalil (Training Systems); Peter Miller (Research Strategy); Georgina Rawle (Financial & Expenditure Study); Dr. Zeba Sathar (Research Strategy & Qualitative Field Work); Dr. Sameen Siddiqi (Systems Analysis); Dr. Catriona Waddington (Technical Team Leader pro-tem); Patrick Ward (Survey Design and Analysis); Philippa Wood (Institutional & Organisational Analysis; Dissemination Strategy). This report is based on results from a Qualitative Study conducted by Dr Fehmida Jalil and from Quantitative Surveys. The Federal Bureau of Statistics provided the control sample. Patrick Ward managed the design and analysis of the surveys. Other members of the OPM evaluation team assisted in the design of the survey and questionnaires, particularly Simon Hunt and Juan Munoz. A team of analysts worked on the analysis and report writing. They were: Shafique Arif, Bianca Camac, Ludovico Carraro, Megan Douthwaite, Simon Hunt, Nils Riemenschneider, Patrick Ward and Philippa Wood. The Population Council, Islamabad, was responsible for field implementation and data processing and provided assistance in the design and piloting of the questionnaires. Shafique Arif was the survey manager for the field operation and Minhaj ul Haque was responsible for data processing. Mehmood Asghar provided support for the desktop publishing. A large team of supervisors, enumerators and others worked on the survey and are listed in the Quantitative Survey Report. The cooperation and assistance provided by staff members of the National Programme for Family Planning and Primary Health Care throughout the country, including the Lady Health Workers and their supervisors, is also gratefully acknowledged, as is the cooperation of the householders, community members and health facility staff interviewed.

TABLE OF CONTENTS ABBREVIATIONS...IV EXECUTIVE SUMMARY...V CHAPTER 1 THE LHWP-PROVIDING SERVICES TO THE DOORSTEP... 1 The Lady Health Worker Programme...2 Lady Health Worker Programme Evaluation...3 Results from the LHW Evaluation...3 Characteristics of the Lady Health Worker and her Supervisor...4 Service Delivery...5 Lady Health Workers Performance...5 The Relationship of Knowledge to Performance...6 Key Points...7 CHAPTER 2 THE KNOWLEDGE SCORE... 9 The Knowledge Test...10 Knowledge Test Results...10 Analysing the Results...13 Scoring well on the Knowledge Test...14 Key Points...16 CHAPTER 3 DELIVERY OF TRAINING... 17 Growth of the Programme...18 Training of Trainers...19 Lady Health Workers Training...20 Training of Lady Health Supervisors...23 Barriers to Effective Training...25 Cost of Training...25 Key Points...27

Tables Table 1-1 Table 2-1 Table 2-2 Table 2-3 Table 3-1 Table 3-2 Age, Education and Marital Status of LHWs and their Supervisors...4 The Knowledge Test-General Knowledge Section. Percentage of Correct Answers Given by LHWs and Supervisors...11 The Knowledge Test-Case Based Questions- Percentage of Correct Answers Given by LHWs and Supervisors...12 Supervisors Knowledge Score by Age Group, Educational Attainment, Duration of Service and Training Received...15 Lady Health Workers Annual Turnover...19 Training of LHWs...22 Table 3-3 Training of LHSs...24 Table 3-4 Unit Cost of a Fully Funded Programme (2000/01 prices)...26 Table A. 3.1 Cascade of Training...33 Table A. 4.1 Sample sizes for Final Analysis...34 Table A. 5.1 Table A. 6.1 Table A. 6.2 Table A. 7.1 Different Levels of Performance amongst Lady Health Workers...36 Scoring for General Knowledge section of the Knowledge Test...38 Scoring for Case-based Section of the Knowledge Test...39 Observation of both the Supervisors and LHWs Training Session...41 Figures Figure 1-1 Lady Health Worker Provision of Service by Performance Category...5 Figure 2-1 Figure 2-2 Knowledge Score for Lady Health Workers and Supervisors...13 Relationship between the Knowledge Score and the Level of Education of LHWs and Supervisors...14 Figure 3-1 Planned Recruitment vs. Actual Recruitment of LHWs 1994/95-1999-98...18 Figure 3-2 Number of Allocated Posts vs. Number of Lady Health Workers Working...19 Figure 3-3 System for Training of Trainers...20 Figure 3-4 Share of Total LHWP Expenditure by Input/Activity- 1998/99 to 2000/01...25 Appendices Appendix 1: Lady Health Worker- Roles and Responsibilities...30 Appendix 2: Lady Health Supervisors Roles and Responsibilities...32 ii

Appendix 3: Training of Trainers...33 Appendix 4: The Quantitative Survey...34 Appendix 5: Levels of Performance...35 Appendix 6: The Knowledge Test...37 Appendix 7: Report on Observation of Training Sessions, Interviews Conducted and Documents Reviewed...41 iii

ABBREVIATIONS AJK ARI BHU CDD CPR DFID DHO DOT DPIU EPI FANA FATA FHT FLCF FPO FPIU GOP HMIS ICT IDD IMCI LHS LHW LHWP MCH MoH MT NAs NGO NWFP PHC POL PSU PPIU RHC THQ TBA WMO Azad Jammu and Kashmir Acute Respiratory Illness Basic Health Unit Control of Diarrhoeal Disease Contraceptive Prevalence Rate Department for International Development (UK) District Health Office Directly Observed Therapy District Programme Implementation Unit Expanded Programme of Immunisation Federally Administered Northern Area Federally Administered Tribal Area Female Health Technician First Level Care Facility Field Programme Officer Federal Programme Implementation Unit Government of Pakistan Health Management Information System Islamabad Capital Territory Iodine Deficiency Disorders Integrated Management of Childhood Infections Lady Health (Worker) Supervisor Lady Health Worker Lady Health Worker Programme Mother and Child Health Ministry of Health Master Trainer Northern Areas (FANA) Non-Government Organisation North Western Frontier Province Primary Health Care Petrol, Oil and Lubrication Primary Sampling Unit Provincial Programme Implementation Unit Rural Health Centre Tehsil Headquarters Hospital Traditional Birth Attendant Woman Medical Officer iv

EXECUTIVE SUMMARY Over the past seven years the Lady Health Worker Programme (LHWP) has become an important element in the Government of Pakistan s plan to raise the health status of women and children in villages and poor urban areas. Over thirty million people are receiving services from a Lady Health Worker (LHW) in their community. The evaluation shows that these services have a positive impact on the health of the poor, particularly women and children. Through their work, LHWs are contributing directly to improved hygiene and higher levels of contraceptive use, iron supplementation, growth monitoring and vaccination amongst their clients. Almost three out of four communities report that the LHWP has generally improved people s lives. This report presents information on the work knowledge of the Lady Health Workers and their Supervisors and the performance of the training system. The training of the LHW is conducted at the Health Facility where she was recruited and where, once trained, she will refer her clients and will meet once a month to hand in her reports, receive her supplies and receive refresher, or additional skills training if it is provided. The trainers of the LHW are the staff of the facility who were trained through a training of trainers system. The training system has low financial costs as it uses existing Ministry of Health and Provincial Health Department staff and takes place at the Health Facility. For the 15 months of the LHWs training, 3 months full-time and then one week a month for 12 months, the trainers will receive an additional 20% allowance. In the past three years the cost of training per Lady Health Worker represents around four percent of the LHW programme budget. The training system should provide LHWs with the essential knowledge and skills to carry out her role in the community. A trained LHW will be able to make a greater contribution to increasing the health knowledge of the community and in changing her clients behaviour. Lady Health Worker knowledge is high in many areas. The average score on the Knowledge Test for LHWs is sixty-nine percent and for Supervisors seventy-four. While LHW knowledge is high in many areas, the lack of in-depth knowledge for a minority of LHWs is of concern. This lack of sufficient depth of knowledge may have serious clinical consequences as well as undermining the professional reputation of the Programme. From the evaluation results we found that LHWs with more knowledge provided a higher level of services. This could be due to intrinsic motivation on the part of the LHW where, if she is dedicated to providing services in her community, she ensures that she has the necessary knowledge. It could also be that a more knowledgeable LHW feels more competent and therefore confident in carrying out her work. From our research the main variable that relates to the knowledge levels of both LHWs and Lady Health Supervisors (LHS) is their level of education. The higher the level of education- the higher the score on the Knowledge Test. 1 Unless the Programme can provide sufficient education to overcome this effect it should continue to recruit LHWs and Supervisors with as high an education level as possible. 1 This was a test based on the LHW curriculum, designed for the purposes of this evaluation to assess LHWs and LHSs knowledge- see Chapter 2 and the Quantitative Survey Report.

The LHWs in our research had all been working for at least four years and had all completed basic training. While they had attended at least one additional training-on a particular topic they had not been involved in regular refresher training. From the Qualitative studies it appears that there is opportunity for the LHS to provide greater coaching during their supervisory visits. Only fifty-one percent of LHWs reported receiving on-the-job training from her Supervisor. The monthly meeting also provides time for increasing knowledge and skills through training- both through lectures and participatory problem solving and through an active programme of testing. Twenty-five percent of LHWs had received training at their last monthly meeting. The programme can support this by providing useful materials and lesson plans designed for such meetings. Given the scale of the operation and the amount of resources available, the training system will always face risks to efficient and effective performance. There are few controls on the quality of the trainers throughout the system. Whilst LHSs monitor that the training is happening, there is no measure of training competence once the trainers have completed their training course and the LHS cannot exert much influence over Medical Officers of the Health Facilities. The 20% training allowance provides some leverage for the Programme over training trainer performance however there is no indication that this leverage is used to gain improved performance. 2 While economical the training of trainers approach increases the risk that the training skills imparted at the Federal level may not trickle down to the Health Facility. Using staff at the Health Facility for training has the big advantage of their developing a working relationship with the LHW, however the downside is that they were not actively recruited to the role of trainer on the basis of competence or motivation. 3 An additional issue, not related particularly to training but important nonetheless, is that the LHW training is for the majority of LHWs their entry into the paid workforce. The working behaviour of the Health Facility staff provides her with a model of performance. However the performance of Health Facility staff throughout the country is very variable and many may not be good role models for the LHW. The implication is that the messages from the Programme about the level of service that a LHW should provide should be reinforced vigorously through other channels, for example, the LHS. The biggest gain to knowledge will probably come through providing the LHW with high quality feedback on her knowledge levels and giving her targets to aim for. The Knowledge Test developed for this evaluation could be used as the basis for developing a suitable test that could be used nationally. The results from the evaluation provide both a benchmark against which LHWs could measure their performance and a realistic base measure on which to develop performance standards. 2 3 Once the 15 months training is completed the training allowance to the three FLCF trainers is stopped. This reportedly results in a number of them losing interest in the Programme and the LHWs. The qualitative studies point to there being greater on-going support for LHWs from the Facility if the staff who conducted the initial training are still posted there. vi

Chapter 1 THE LHWP-PROVIDING SERVICES TO THE DOORSTEP IN THIS CHAPTER: The Lady Health Worker Programme Lady Health Worker Programme Evaluation Characteristics of the LHW and her Supervisor Key Points

TRAINING PROGRAMME REVIEW THE LADY HEALTH WORKER PROGRAMME The Lady Health Worker Programme (LHWP) has become an important element in the Government of Pakistan s plan to raise the health status of women and children in rural villages and poor urban areas. The Programme was conceived in 1993 and launched in April 1994 as a Federal development programme funded by the Ministry of Health (MoH), and implemented by both the MoH and the provincial Departments of Health. The main goal of the programme was to establish a primary health care service: providing accessible promotive, preventive, curative and rehabilitative services to the entire population bringing about community participation improving the utilisation of health facilities expanding availability of family planning services in urban slums and rural areas of Pakistan, and gradually integrating existing health care delivery programmes like EPI, Malaria control, Nutrition, MCH within the programme. 4 A key Programme objective was to bridge the gap between service delivery from health facilities and the community, in rural and less developed urban areas, by creating a new cadre of community workers - the LHWs. Each LHW would be trained to provide basic primary health care services and family planning services to around 200 families living in the area adjoining her residence. 5 Figure 1-1 Organisational Structureof the LHWP FPIU MoH Programme Implementation Units were to be established at the Federal, Provincial and District level in the Ministry and Departments of Health, to ensure effective implementation and on-going support for the LHW- including recruitment, payment of salaries, supervision, supplies and training (Figure 1-1). 6 PPIU DPIU LHS LHW DoH DHO Health Facility Training of the LHW was to be provided by personnel at the health facility. Depending on the staff available this team would include a Medical Officer, a Lady Health Visitor (LHV) and a Health Technician. A twenty percent training allowance was to be paid to these staff throughout the fifteen months it takes to complete the training. As well as being responsible for regular training they would also ensure a formal link between the LHW and the local health facility. The LHV was to be responsible for ongoing supervision. In practice this plan has generally been implemented, except that the LHVs were not available for supervision and instead a separate cadre of supervisors was established. 4 5 6 Ministry of Health, Government of Pakistan Prime Minister s Programme for Family Planning and Primary Health Care- Revised PC-1, August 1995- page 3. See Appendix Two for Lady Health Workers responsibilities. See the Final Report, Quantitative Survey Report and the Provincial Survey Reports for information and evaluation of organisational systems. 2

THE LHWP-PROVIDING SERVICES TO THE DOORSTEP LADY HEALTH WORKER PROGRAMME EVALUATION In March 1999 an external evaluation was begun of the LHWP by Oxford Policy Management funded by the Department for International Development (DFID) UK. 7 This report presents information, collected as part of this external evaluation through quantitative surveys and a qualitative study of training sessions, on the knowledge of the LHW and her supervisor and on the performance of the training system. 8 It covers: the relationship of levels of LHW knowledge with their level of performance the knowledge and skills levels that the LHW and her supervisor bring to their work information on training provision to LHWs and their Supervisors unit cost of training LHWs This information should support programme managers in identifying initiatives to improve programme performance through raising the knowledge and skills levels of LHWs and their Supervisors. RESULTS FROM THE LHW EVALUATION Our survey shows that over thirty million people in Pakistan are receiving services from the LHWP 9 and that these services are making a difference to health behaviour and health outcomes. LHWs are providing health services to the poor, particularly women and children, which contribute directly to higher levels of contraceptive use, vaccination in children, better hygiene behaviour, iron supplementation, antenatal care and growth monitoring amongst their clients. 10 The range of services that LHWs provide to their clients include: Hygiene education on drinking water and sanitation Nutritional advice and growth monitoring Monitoring and advising women on their health, and that of their babies Motivating and educating women on family planning Promoting and facilitating vaccination 7 This report is one of a series of reports providing the results of the evaluation. There are three national level reports, the Final Report which addresses key policy, service delivery and management issues; the Quantitative Survey Report providing an extensive analysis of the quantitative results; and the Financial and Economic Analysis presenting costs and spending patterns of the LHWP. In addition to these reports there are four white cover reports: the Training Programme Review (this report), the Qualitative Report, the Survey Sampling Design and the Quantitative Survey Questionnaires. 8 A separate note reviewing the training manuals in detail has been provided to the programme management. 9 See Quantitative Survey Report for calculation on number of LHWs in the field. 10 See Quantitative Survey Report for information on the impact of the LHW on health outcomes. 3

TRAINING PROGRAMME REVIEW CHARACTERISTICS OF THE LADY HEALTH WORKER AND HER SUPERVISOR The LHWP has specific recruitment criteria for Lady Health Workers and their Supervisors which are designed to ensure that the LHW has the capacity to provide a professional and safe service. As can be seen from the Quantitative Survey results below, the Programme has been successful in meeting these criteria (Table 1-1). Age and Marital Status: LHWs should be between twenty and fifty years old, though if married, eighteen and nineteen-year-olds are acceptable. Only a small fraction of LHWs did not meet this criteria. The majority of LHWs are currently married, as preferred by the Programme, although almost a third have never been married. Three quarters of the Lady Health Supervisors (LHS) are in their twenties and slightly over half are married. Education: LHWs should be educated to at least the eighth class, though it is preferable for them to have matriculated. Based on LHW self-reporting, only about 2 per cent have received less than 8 years of education, though 40 per cent reported receiving less than the ten years required for matriculation. However, only three quarters of LHWs could prove their education with a school certificate. While the selection criteria for supervisors require at least an intermediate (class 12 pass), in practice most supervisors are considerably better qualified. More than half of the supervisors are graduates or post-graduates. LHW Residence: The vast majority of LHWs live in the village/mohalla in which they work, as required by Programme standards. Nationally four percent are non-residents. Table 1-1 Age, Education and Marital Status of LHWs and their Supervisors Characteristic LHWs (%) LHSs (%) Age Distribution 15 19 years 1 20 24 years 20 27 25 29 years 41 50 30 34 years 15 16 35 39 years 10 4 40 years and above 13 4 Mean age 29.6 28 Education Less than 8 years 2 0 8 or 9 years 38 0 Matric 50 4 Intermediate 9 41 Graduate 2 39 Post graduate 15 Marital Status Never married 28 46 Currently married 62 52 Widow/divorced/separated 10 2 4

THE LHWP-PROVIDING SERVICES TO THE DOORSTEP SERVICE DELIVERY As mentioned earlier, over thirty million people across the country are receiving services from a LHW. Overall the services the LHWP has been providing for the past seven years are having a positive impact on health amongst the poor and particularly women and children. LHWs are contributing directly to higher levels of contraceptive use, iron supplementation, antenatal care, growth monitoring and vaccinations amongst their clients. The level of service varies according to the type of service provided. Nationally, vaccination promotion reaches the largest proportion of LHW clients. However, at the other end of the scale, LHWs undertake very little of the growth monitoring they are supposed to provide. LHWs report that curative services are an important part of their workload and that many of them see patients with emergencies, sometimes of a serious nature. Most client communities have a positive perception of their LHW. Clinical support services for the LHWs at the health facilities are inadequate and would be expected to reduce the effectiveness of the LHWs referral role. LADY HEALTH WORKERS PERFORMANCE We have developed a performance measure of LHW service delivery using a selection of ten preventive services. 11 These cover LHW activities in, hygiene, health education, vaccination promotion, family planning, pregnancy and birth, child nutrition and growth monitoring. Using our measure of performance we find major differences in the levels of service delivery amongst LHWs. The top 25% - the High Performers provide significantly more services than the bottom 25%- the Poor Performers. In between we have the Good Performers and those who are Below Average. Figure 1-2 Lady Health Worker Provision of Service by Performance Category % target population provided services 100 90 80 70 60 50 40 30 20 10 0 17 Poor Performers 35 Below Average Performers 48 Good Performers 68 High Performers 11 See Appendix 4 and the Quantitative Survey Report for a description of the performance measure of LHW service delivery. 5

TRAINING PROGRAMME REVIEW On our performance measure the Poor Performers are on average only providing services to seventeen percent of their eligible clients as compared to the High Performers who provide on average services to sixty-eight percent (Figure 1-2) It is easy to distinguish Poor Performers because they fail to deliver across the whole range of services whereas High Performers cover at least sixty percent of clients 12 - and often well above this - for all services, except growth monitoring. 13 High Performing LHWs are also working longer hours. In the week preceding the survey they had worked over twenty-two hours compared to seventeen for poor performers. THE RELATIONSHIP OF KNOWLEDGE TO PERFORMANCE We tested many factors to identify those that explain the large variations in LHW performance. 14 We found that nationally, High Performers have a higher level of knowledge, are better supervised and are better supplied with medicines and equipment. The supervisors of High Performers also have higher levels of knowledge, are more likely to have been fully trained and have access to a programme vehicle for field supervision. The Evaluation Team found that it would be possible to achieve a significant improvement in service delivery if Supervisors were provided with transport and if LHW scores on the Knowledge Test 15 were improved to an average of eighty percent. We do not have performance information by District from the Quantitative Surveys, however stakeholder consultations, the qualitative studies and key informant interviews point to the importance of the District Health Management in the ensuring the effective functioning of the Programme. Particularly in the area of training, a good District Co-ordinator will be able to support high levels of performance from LHWs by being active in ensuring high quality training is provided for LHWs and their Supervisors both at the District and Facility level so that transfer of knowledge from the classroom to the field is supported through good supervision. 12 See Appendix 5 and the Quantitative Survey Report for more information on different levels of performance amongst Lady Health Workers. 13 The low level of growth monitoring even in the relatively high performing LHWs suggests that there are specific problems that need to be addressed if the Programme considers it important to offer this service. 14 For a full explanation of the analysis see the Quantitative Survey Report. 15 See Chapter 5. High Performers have greater knowledge and provide a greater level of service coverage. 6

THE LHWP-PROVIDING SERVICES TO THE DOORSTEP KEY POINTS The LHWP has become an important element in the Government of Pakistan s plan to raise the health status of women and children in villages and poor urban areas. This report presents information, collected through quantitative surveys, qualitative studies and financial analysis on the knowledge of the LHW and the performance of the training system. The Programme has been successful in ensuring the vast majority of LHWs and their Supervisors meet the educational, age and residency criteria. 7

Chapter 2 THE KNOWLEDGE SCORE IN THIS CHAPTER: The Knowledge Test Knowledge Test Results Analysing the Results Scoring Well on the Knowledge Test Key Points

TRAINING PROGRAMME REVIEW THE KNOWLEDGE TEST LHWs and their Supervisors who participated in the Quantitative Surveys were tested using the Knowledge Test. 16 The Knowledge Test is divided into two sections: general questions covering a range of preventive and curative health care issues, case histories where the problem must be identified and responded to with the treatment or advice that would be provided to the patient. A Knowledge Score was arrived at on the basis of how many questions were answered correctly. The score is presented as the percentage of correct answers given against the total number of questions. 17 KNOWLEDGE TEST RESULTS The average score for LHWs was sixty-nine percent and for Supervisors seventy-four. The results for Lady Health Workers and their Supervisors- are presented below. These include the results for the general knowledge (Table 2-1) and the case based results (Table 2-2) 16 This is a test developed by the Evaluation team to assess LHWs and LHSs work-related knowledge and skills. 17 See Appendices 7 and 8 and the Quantitative Survey Report for further information on the Knowledge Test and the results. 10

THE KNOWLEDGE SCORE Table 2-1 The Knowledge Test-General Knowledge Section. Percentage of Correct Answers Given by LHWs and Supervisors Measure LHWs % LHSs % Contraindications for the contraceptive pill % giving at least one correct answer 93 100 % giving 3 or more correct answers 50 88 Contraindications for the IUD % giving at least one correct answer 76 98 % giving 3 or more correct answers 5 36 Breastfeeding and nutrition % giving at least one correct response about breast feeding 99 100 % giving three or more correct responses about breast feeding 65 67 % stating that mothers should start breastfeeding within four hours of birth 98 100 % stating that mothers should feed baby the colostrum 96 100 % stating that weaning foods should be introduced at the age of 4-6 months 88 98 EPI vaccination schedule % who could name all four vaccines (BCG, DPT, Polio, Measles) 94 98 % identifying all four vaccines and giving correct number of doses 73 83 % identifying all four vaccines and giving the number of doses and the correct ages for each dose 43 59 Diarrhoea % giving at least one correct answer to mother of child with diarrhoea and mild 100 100 dehydration, if lacking packets of ORS % giving three or more correct answers to mother of child with diarrhoea and 58 81 mild dehydration, if lacking packets of ORS % giving at least one correct answer to mother of a child that will not take ORS 85 100 % giving three correct answers to mother of child that will not take ORS 20 38 % able to give at least one correct response on how to prevent diarrhoea 96 100 % able to give three or more correct responses on how to prevent diarrhoea 53 78 Malaria % giving correct answer on how malaria is caught 92 97 % saying they would give Chloroquine 56 61 % saying they would refer to a health facility 80 88 % giving correct dose of Chloroquine 6 10 HIV % giving at least one correct response on how HIV is transmitted 93 100 % giving three or more correct responses on how HIV is transmitted 20 70 11

TRAINING PROGRAMME REVIEW Table 2-2 The Knowledge Test-Case Based Questions- Percentage of Correct Answers Given by LHWs and Supervisors Measure LHWs % LHSs% Growth Monitoring Card Case 1-Normal to Moderate Malnutrition % giving correct weight of child 39 75 % saying that the child is normal or moderately malnourished 68 64 % stating that the child is growing adequately 68 76 % correctly stating that referral is not necessary 78 86 Case 2-Severely Malnourished % giving correct weight of child 68 79 % saying that the child is severely malnourished 40 93 % stating that the child is failing to gain weight 30 50 % correctly stating that referral is necessary 68 75 % requesting information about eating and feeding practices 81 90 % requesting information about recent illnesses 62 73 % requesting information about eating and feeding practices and recent illnesses 56 69 Diarrhoea/Dehydration Management Case 1-Mild to Moderate Dehydration % stating that the child is mild to moderately dehydrated 75 88 % stating that the child should be rehydrated (ORS or SSS) 84 83 % stating that the child should be breast fed more often 77 84 % stating that the child should be rehydrated (ORS or SSS) or breast fed more 94 98 often % stating that the child should be brought back soon for reassessment 1 8 % stating that the parents should seek help soon if the child does not improve 9 15 % correctly stating that referral is not necessary 38 53 Case 2-Severe Dehydration % stating that the child is severely dehydrated 65 63 % stating that the child should be rehydrated (ORS or SSS) or breast fed more 86 76 often % stating that the child should be rehydrated (ORS or SSS) or breast fed more 58 61 often and referred to a health centre % stating that the child should be referred to a health centre 84 84 Respiratory Infections Case 1-Severe Pneumonia % identifying severe/very severe pneumonia 71 68 % stating that the child should be referred to a health centre 89 85 % stating that the child should be given antibiotics 84 87 % stating that the child should be given a single dose of antibiotics and referred 74 75 Case 2-Simple Pneumonia % identifying simple pneumonia 21 28 % stating that they would give antibiotics 85 77 % stating that the child should be given fluids or breast fed more often 42 51 % stating that the child should be watched for danger signs 6 11 % stating that they would give a full course of Cotrimoxazole (of all 62 50 LHWs/Supervisors) % stating correct dose & duration of Cotrimoxazole course (of those who would give a full course) 23 31 12

THE KNOWLEDGE SCORE Measure LHWs % LHSs% Pregnancy Case 1-Anaemia % identifying anaemia 93 91 % stating that they would examine the patient s conjunctiva/eye for anaemia 77 82 % stating that they would examine the patient s conjunctiva/eye for anaemia, ask 16 19 about the patient s diet and ask about recent illnesses % stating that they would give iron tablets (Fefan) 80 72 % stating that they would advise the patient to eat iron-containing foods 78 87 % stating that they would advise the patient to avoid heavy work and to rest 15 18 Case 2-Pre-eclampsia % stating that they would advise the patient have her blood pressure measured 12 10 % stating that they would refer to a health centre and that the referral would be urgent 61 61 * Number of LHSs in sample too small for reliable estimates. The results vary across the provinces. For LHWs, Sindh has the highest average score and for Supervisors it is AJK and Northern Areas (Figure 2-1) The Programme in Sindh has reportedly been conducting more testing of LHWs than other provinces and this could explain the better results. Figure 2-1 Knowledge Score for Lady Health Workers and Supervisors 100 % Knowledge Score 90 80 70 60 50 40 74 75 75 70 71 68 60 63 69 77 LHW Supervisor 30 Punjab & ICT Sindh NWFP & FATA Balochistan AJK & NAs Source: The Quantitative Survey Report. ANALYSING THE RESULTS The level of general clinical knowledge of LHWs and their supervisors is reasonably good. This knowledge however may lack depth, as is shown by the drop in their ability to provide multiple correct responses. There were also some specific areas of weakness. One of these is knowledge of the vaccination schedule. Only forty-three percent LHWs (fifty-nine percent of their supervisors) could name the four EPI vaccines, give the correct number of doses and the correct age at which doses are given. 18 18 This problem is largely caused by DPT and Polio, as there are a large number of doses and ages to remember. 13

TRAINING PROGRAMME REVIEW There are also serious deficiencies in the ability of LHWs to provide the correct doses of medicines required in basic situations. Only six percent of LHWs (ten percent of Supervisors) were able to provide the correct dose of Chloroquine to give to a child with symptoms of malaria, even though they were encouraged to use the Programme manual or medicine box to answer the question. Performance is much poorer on the case history-based questions. For many questions over thirty percent of the LHWs, throughout the country, could not give the correct answer. LHWs ability to read and interpret child growth cards is also deficient. Only thirty-nine percent of LHWs (seventy-five percent of supervisors) were able to provide the correct weight of a normal/moderately malnourished child and only thirty percent (fifty percent of supervisors) were able to assess that a severely malnourished child was failing to gain weight. These results are poor nationwide and imply that only about thirty to forty percent of LHWs across the country are able to conduct meaningful child growth monitoring. The lack of knowledge of a minority of LHWs could have serious clinical consequences to their clients. This needs to be addressed by the Programme. SCORING WELL ON THE KNOWLEDGE TEST For the LHW, in general, the higher the educational level, the higher will be her score on the Knowledge Test (Figure 2-2). Figure 2-2 Relationship between the Knowledge Score and the Level of Education of LHWs and Supervisors 100 90 Knowledge score 80 70 60 50 59 69 70 71 72 75 73 74 75 LHW Supervisor 40 30 < 8 yrs 8-9 yrs Matric Intermediate Graduate Post Grad Highest level of education The average score for a LHW with less than eight years of education was fifty-nine percent while that of an LHW educated to intermediate level was seventy-two percent. As the 14

THE KNOWLEDGE SCORE level of knowledge is an important factor in the LHW s ability to provide quality services, these results suggest that the Programme should maintain its educational criterion for LHWs. We also found that if the LHW had been provided with specific training on growth monitoring or immunisation, then this was associated with better score on the Knowledge Test, not only on the particular topic she had been trained in, but overall. The mean score for LHSs was tabulated against a number of other variables. While some variables showed marked correlation with their scores, others did not. Variables that failed to show a relationship include whether or not the supervisor received initial training (only a small number of cases), the number of months of initial training and the Supervisor s age (Table 2-3). Education, training and experience on the job were important factors contributing to a Supervisor gaining a high score. 19 Supervisors who were fully trained scored seventy-five percent on average whereas those who had attended only the initial training and not the inservice training scored sixty-five percent. Table 2-3 Supervisors Knowledge Score by Age Group, Educational Attainment, Duration of Service and Training Received Measure Mean score Age group 20 24 71 25 29 75 30 34 74 35 39 75 40 and above 70 Duration of service Under one year 67 1 2 years 70 2 3 years 73 More than 3 years 75 Training received All required training 75 Initial and one of part times 74 Only initial 65 None (1) 68 Note: (1) Based on a small number of cases (10). This contrasts with the LHWs, where variations in training seem to have little relationship with knowledge scores. This may well be because the supervisors are better educated to begin with and therefore benefit more from the training received. Indeed, Figure 2-2 suggests that the difference between LHW and supervisors knowledge levels is mostly a result of differences in their levels of education: for similar levels of education, LHWs and supervisors have very similar scores. 19 Experience could also be important for LHWs. Because our sample are all experienced LHWs this factor does not stand out as a variable that makes a difference to their knowledge. 15

TRAINING PROGRAMME REVIEW The level of clinical knowledge of LHWs and their Supervisors is important in the provision of a professional and safe service and for the reputation of the LHWP. In addition we have found that LHWs who deliver more services- the High Performers- have higher Knowledge Score. This could be due to intrinsic motivation on the part of the LHW where if she is dedicated to providing services in her community she ensures that she has the necessary knowledge. It could also be that a more knowledgeable LHW feels more competent and therefore confident in carrying out her work. The main means of increasing the knowledge of the current LHWs and Supervisors will be through training and on-the-job coaching. KEY POINTS Lady Health Worker knowledge is high in many areas. The average score on the Knowledge Test for LHWs is sixty-nine percent and for Supervisors seventy-four. While LHW knowledge is high in many areas, the lack of in-depth knowledge for a minority of LHWs is of concern. This lack of sufficient depth of knowledge may have serious clinical consequences as well as undermining the professional reputation of the Programme. In addition we have found that LHWs who deliver more services- High Performers- have higher scores on the Knowledge Test. The Programme must aim to increase the knowledge of both LHWs and their supervisors. The level of education of Lady Health Workers and Supervisors is correlated with their Knowledge Score. The higher the education, the higher the score. It is important that the Programme does not lower its education criteria if it wants to maintain performance. 16

Chapter 3 DELIVERY OF TRAINING IN THIS CHAPTER: Growth of the Programme Training of Trainers Lady Health Workers Training Lady Health Supervisors Training Cost of Training Key Points

TRAINING PROGRAMME REVIEW GROWTH OF THE PROGRAMME The program originally planned to train 100,000 LHWs over five years (1993-98), covering all villages in Pakistan. Between 1994/95-1997/98 the actual recruitment of LHWs was less than fifty percent of the target envisaged in the R-PC1 (Figure 3-1). 20 In the first phase in FY1994/95, approximately 10,000 LHWs were recruited. In Phase Two in FY1995/96 there were approximately 14,000 LHW recruited. In Phase Three in 1996/97 approximately 20,000 LHW were recruited. By 1997/98 only 45,000 LHW posts had been sanctioned. Between 1997/98 and 1998/99 there was an extension of Phase Three where LHWs who had either dropped out of the programme or had their appointments terminated were replaced. Approximately 10,000 LHW have been recruited in the extended Phase Three period. This represents an annual turnover of approximately six percent. Figure 3-1 Planned Recruitment vs. Actual Recruitment of LHWs 1994/95-1999-98 120000 100000 No. of LHWs 80000 60000 40000 Planned Actual 20000 0 1994/95 1995/96 1996/97 1997/98 Source: Planned recruitment, as per the R-PC1, and Actual recruitment as per the LHW Database- FPIU, LHWP. There was no new recruitment in the year 1999/00. In 2000/01, selection began for Phase Four in which an additional 13,000 posts were allocated bringing the total number of allocated positions to 58,000. Unfortunately procurement delays have hindered the training of the Phase Four recruits, many of whom are currently (February 2002) undergoing the initial 3-month training. The actual numbers of LHWs working since 1997/98 are shown in Figure 3-2 and Table 3-1. The steady decline reflects an annual turnover of approximately five percent, similar to that recorded for the first three phases. By June 2001, 37,838 LHW were recorded on the payroll. 21 20 See the Financial and Economic Report. 21 A variety of sources have been used to collect information on LHW and LHS numbers, including the FPIU database, the HMIS system and the payroll. Details are shown in the Financial and Economic Report. 18

DELIVERY OF TRAINING Figure 3-2 Number of Allocated Posts vs. Number of Lady Health Workers Working 70,000 60,000 50,000 # of LHWs 40,000 30,000 20,000 10,000 Posts Working at end of FY - 1997/98 1998/99 1999/00 2000/01 Table 3-1 Lady Health Workers Annual Turnover LHW 1997/98 1998/99 1999/00 2000/01 Posts 45,000 45,000 45,000 58,000 Working at end of FY 43,963 41,492 39,665 37,838 Annual turnover 6% 4% 5% Source of LHW data FPIU LHW database Estimate* Payroll count * Estimate is calculated as the mean of the 1998/99 and 2000/01 figure. TRAINING OF TRAINERS At the beginning of Phase One of the Programme, Master Trainers were trained at the Federal and Provincial level (Figure 3-3). 22 Using a standardised training manual they then trained three to four trainers per District 23, who in turn, trained three or more staff at each Health Facility. These staff were then responsible for the training of the LHW. 24 The DHO is responsible for ensuring that there is least a two staff trained and available at the Health Facility before training starts. If there are no trainers available at the Facility then someone may be moved in from another Health Facility from within the district. Training of new recruits is conducted in batches at the health facility. Training is not provided for individuals. If necessary accommodation is arranged for the LHW, preferably at the Health Facility or as a house guest in the community. Sometimes this is also done for an accompanying person. 22 See Appendix Four. 23 The district training teams comprised: Assistant DHO, Medical officer, WMO, Senior Technician, Assistant Inspector Health Services, Senior LHV and Senior Nurse at DHQ. 24 It was expected that the Medical Officer and Female Health Technician would take primary responsibility for training components of community organisation, EPI, CDD ARI and management of minor ailments, while the Lady Health Visitor would be responsible for the maternal health and family planning component. 19

TRAINING PROGRAMME REVIEW Figure 3-3 System for Training of Trainers National Trainers to Train Provincial Master Trainers Provincial Master Trainers to Train District Master Trainers District Master Trainers to Train Health Facility Trainers Health Facility Trainers to Train LHWs LADY HEALTH WORKERS TRAINING The training system is designed to provide LHWs with: Three months full time basic training for each LHW at the health facility where she was recruited. Twelve months task based (in-service) training, comprising one week full-time per month for twelve months based at the health facility. In addition, training can be provided, though not on a universal and compulsory basis, via: Additional specialist short courses that either offer a reinforcement to the basic training course, or alternatively, provide training in new areas; Training given in monthly health facility meetings; Training by supervisor in one-to-one monthly supervisory meetings. The initial training is designed to provide the foundation of knowledge needed for the LHWs work. Training covers the major PHC intervention areas in nine units. Topics include: recognition and management of childhood illnesses; problems of reproductive age period, family planning, immunisation, nutrition, malaria, skin & eye problems, community organisation and interpersonal communication. The training is conducted five days a week with teaching in the class room for 3 ½ - 4 hours, followed by clinical practice in the outpatient section of the facility for 1 ½ - 2 hours. The LHW is trained to treat the most common conditions affecting the community. The LHW receives a kit with about 10 medicines including contraceptive pills and condoms, growth cards, a chart for checking vision (E charts) and six health education posters for her Health House. She is also given a copy of the training manual for reference. The second stage of training is designed to develop competence in the field. The focus is on the LHWs scope of work, the problems that they have to solve and their ability to carry out specific tasks. It includes three weeks of fieldwork each month- under supervision, followed by one week back in the classroom problem-solving and revising the manual. 20

DELIVERY OF TRAINING Refresher training was not budgeted for in the R-PC1. Reinforcement of knowledge and skills is supposed to occur through the supervision system both the supervisors and the monthly meetings. There is no formal system of evaluation of LHWs knowledge and skills either after the initial training or at the end of their training programme. Once in the field LHWs are evaluated by their supervisors using a monthly performance checklist, a copy of which is sent to DPIU. LHWs who scored less than sixty percent using the checklist are supposed to be given on-the-job training by the Supervisor with further training if necessary by the Health Facility doctor. If her performance scores show no improvement then she may lose her contract. Results from the Quantitative Survey for LHWs Table 3-2 shows the LHWs experience of training and illustrates several important points: All LHWs have received the full-time three month basic training course and ninety-four percent have received at least some in-service training; The Health Facility staff who most usually provide training are male doctors, LHVs and dispensers; The vast majority of LHWs have received some type of additional training. Training that reinforces the skills acquired in their basic training courses being the most common. Seventy percent of LHWs had received further growth monitoring training; Monthly meetings at health facilities should be an important source of training for many LHWs. Almost two thirds have received at least some training at their facility and a quarter reported receiving training at the last monthly meeting. 21

TRAINING PROGRAMME REVIEW Table 3-2 Training of LHWs Measure Value Proportion of LHWs who received initial (basic) training 100 Duration of initial training Less than two months 0 Two months 1 Three months 94 More than three months 5 Total 100 Mean number of months of initial training 3.1 Training was imparted by (1) Medical doctor (male) 87 Medical Doctor (female) 20 Lady Health Visitor 70 Dispenser 48 Male medical health technician 20 Female medical health technician 8 Others 3 Percentage of LHWs who attended part-time training of one week per month 94 Mean months of part-time training attended 12.4 Percentage of LHWs who received additional training Any additional training 95 Growth monitoring training 70 HMIS training 71 Iodized salt training 77 Immunization day training 86 DOTS (treating tuberculosis) training 36 On-the-job training by supervisor 51 Continued training at the health facility 63 Training at the last monthly meeting 25 Note: (1) LHWs listed all staff members who gave training and the panel sums to over 100 percent because of multiple responses. Despite the reasonably high levels of initial training that was provided, as we saw in Chapter Two, there continues to be substantial gaps in LHWs knowledge across the country. This suggests problems in the quality of the training and/or a failure to evaluate and reinforce the knowledge imparted. It may be in some cases that the trainers themselves did not have sufficient knowledge, or lacked teaching skills particularly in providing experiential training. Refresher and on-the-job training needs to be readily available in order to maintain and update knowledge. We found that further training in growth monitoring and for immunisation days has been shown to improve LHWs knowledge overall, but this is not the case with other types of additional training she has received - which suggest the quality of training needs to be improved. Only fifty-one percent of LHWs reported receiving on-the-job training from their Supervisors. We found that there was no relationship between the LHWs knowledge score and the level of supervision she received. 25 It appears likely that the style of supervision being provided is not reinforcing the LHWs skills and knowledge. It is important that supervisors 25 See Quantitative Survey Report. 22