Lady Health Worker Programme

Transcription

1 Lady Health Worker Programme External Evaluation of the National Programme for Family Planning and Primary Health Care Summary of Final Report Oxford Policy Management March 2002

2 Reports from this Evaluation 1. Final Report 2. Quantitative Survey Report 3. Financial and Economic Analysis 4. Punjab and ICT Survey Report 5. Sindh Survey Report 6. NWFP and FATA Survey Report 7. Balochistan Survey Report 8. AJK and FANA Survey Report White Cover Reports 9. Training Programme Review 10. Qualitative Report 11. Survey Sampling Design 12. Quantitative Survey Questionnaires

3 ACKNOWLEDGEMENTS This evaluation of the National Programme for Family Planning and Primary Health Care was undertaken by Oxford Policy Management, UK, at the request of the Programme. The evaluation has received very considerable support from the Federal Ministry of Health and from the UK Department for International Development. The cooperation and assistance provided by staff members of the National Programme for Family Planning and Primary Health Care, including the Lady Health Workers and their supervisors, is also gratefully acknowledged, as is the cooperation of the householders, community members and health facility staff interviewed.

4

5 TABLE OF CONTENTS ABBREVIATIONS... III EXECUTIVE SUMMARY...V Introduction...v LHWP Design...v Evaluation Approach...vi Main Evaluation Conclusions... vii Key Points from other Evaluation Reports... viii 1. INTRODUCTION: THE LADY HEALTH WORKER PROGRAMME... 1 Primary Health Care in Pakistan...1 Challenges in Establishing a Community Health Worker Programme...1 The LHWP ISSUES AND SCOPE OF THE EVALUATION... 7 The Structure of this Evaluation...7 Evaluation Approach THE SERVICES PROVIDED BY THE LHW... 9 Introduction...9 The Technical Mix of Services...9 Population Coverage...9 Preventative and Promotional Services...11 Curative Services...13 High and Low Performing LHWs...15 Providing a Pro-poor Service...17 The LHW Poverty Focus LADY HEALTH WORKERS SUPPORT SERVICES...21 Recruitment and Training...21 The Quality of Services LHW s Knowledge...22 Supplies and Equipment...23 Salary Payments...25 Supervision & Vehicles...25 FLCF Support LHW IMPACT ON HEALTH...29 Impact on Health Status...29 Methodological Issues...30 Crude Comparison of Health Indicators between LHW and Control Groups...30 Impact from Direct Provision and Referral...32 Impact from Health Promotion Activities...33

6 Impact Measured by Using Time Trends...34 Impact Measured by Using Statistical Models...35 The Evidence for Fertility and Infant Mortality Impacts COSTS OF SERVICE PROVISION...37 Introduction...37 Actual Expenditure and Actual Unit Costs...37 The Unit Costs of a Fully Funded Programme...40 The Aggregate Costs of a Fully Funded Programme KEY ISSUES AND JUDGEMENTS...43 Main Conclusions...43 Issues for Policy Makers...44 Issues for Programme Managers...45 Tables Table 3.1 Number of household visits made during the preceding week Table 4.1 LHW stock of medicines and other materials...24 Table 4.2 LHW supervisors access to programme vehicles and POL received Table 5.1 Comparison of health measures in LHW and control areas Table 5.2 Comparison of households that received LHW interventions with households that did not receive them (LHW sample only) Table 6.1 Planned and actual unit cost per LHW Table 6.2 Unit cost of an appropriately funded programme (2000/01 prices) Table 6.3 LHWP expansion in the context of Pakistan government health spending Figures Figure 1-1 Management structure of the LHWP... 4 Figure 4-1 Distribution of LHW knowledge score Figure 6-1 Unit cost per LHW planned and real (2000/01 prices) Figure 6-2 Recurrent funding gap ii

7 ABBREVIATIONS AJK Azad Jammu and Kashmir BHU Basic Health Unit CHW Community Health Worker CHWP Community Health Worker Programme EPI Expanded Programme of Immunisation FANA Federally Administered Northern Areas FATA Federally Administered Tribal Areas FLCF First Level Care Facility FPO Field Programme Officer FPIU Federal Programme Implementation Unit GOP Government of Pakistan HMIS Health Management Information System ICT Islamabad Capital Territory LHS Lady Health (Worker) Supervisor LHW Lady Health Worker LHWP Lady Health Worker Programme MCH Mother and Child Health MoH Ministry of Health NWFP North Western Frontier Province ORS oral rehydration solutions PHC Primary Health Care PMP-FPPHC Prime Minister's Programme for Family Planning and Primary Health Care POL Petrol, Oil and Lubrication PPIU Provincial Programme Implementation Unit RHC Rural Health Centre iii

8

9 EXECUTIVE SUMMARY INTRODUCTION The Lady Health Worker Programme (LHWP) was conceived in 1993 and launched in April 1994 as a Federal development programme. This paper provides a summary of the Final Report of the LHWP Evaluation conducted by Oxford Policy Management (OPM). The Evaluation was commissioned by the LHWP in 1999 and completed in The resources available to this project have allowed a comprehensive evaluation using a wide range of research instruments 1 in order to investigate the performance of the LHWP. This summary paper is complemented by twelve reports that have been produced as part of a package of final products. The core reports (seven reports in coloured covers) cover the detailed results of the quantitative surveys, both at the national and regional level; and the financial and economic analysis. In addition, a series of white cover reports are available including: a summary report of the qualitative research and systems study; a report on the training appropriateness review; a report on the sampling strategy; and a folder providing copies of the questionnaires used in the quantitative surveys. LHWP DESIGN In general, Community Health Worker Programmes (CHWPs) such as the LHWP have failed to reach the expectation of providing services with high health impacts, at low cost to an underserved or unserved population. The LHWP, in both its objectives and design, has attempted to overcome the weaknesses that have been found in CHWPs around the world. The LHWP aims to: Address the primary health care problems of the community, providing promotive, preventive, curative and rehabilitative services to which the entire population has effective access, Bring about community participation through creation of awareness, changing of attitudes, organisation and mobilisation of support, Improve the utilisation of health facilities by bridging the gap between the community and health services in the country through Lady Health Workers Expand the family planning services availability in urban slums and rural areas 1 The evaluation undertook three major pieces of fieldwork, the results of which underpin the analysis of the Programme s performance and impact: Quantitative Surveys that provide estimates of performance for each province, AJK and FANA Qualitative Research, and a Systems Study In addition, key informant interviews with more than 60 senior programme staff, Ministry of Health officials and national experts and observers of public health care in Pakistan; participatory and qualitative research in four districts; two major reviews of programme expenditure and financial performance; a systems study in three districts; and a review of training appropriateness.

10 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT Gradually integrate existing health care delivery programmes like EPI, Malaria control, Nutrition, MCH within the Prime Ministers Programme. 2 LHWs are contract workers who live and work in the communities that they serve. LHWs are provided with both full-time and in-service training 3 ; and they are supervised by LHSs and provided with support from nearby First Level Care Facilities (FLCFs). At its inception the LHWP aimed to have 100,000 LHWs in post, covering 100,000 communities and 100 million individuals. The target programme size was to be reached by 1998 at a cost of Rs.9.1 billion. Capital costs are not a large part of the total budget (primarily vehicles for supervisors and training materials). The main costs are recurrent and are made up of 40 percent drugs and contraceptives, 31 percent for LHW salaries, 13 percent for LHS salaries, and 12 percent for LHW training and LHW kit. The Programme is managed by the Ministry and Departments of Health. LHWs and LHSs are recruited on contract, but staff in the Federal, Provincial and District Implementation Units (FPIU, PPIU, DPIUs) are deputed/seconded from the Federal Ministry of Health and the Provincial Department s of Health. Key policy and budget decisions are taken by the Secretary of Health in conjunction with the Planning and Development Division and the FPIU. Most operational decisions are taken by the Provincial and District Programme Implementation Units. EVALUATION APPROACH This evaluation is essentially about whether the LHWP has been able to buck both the international and national trend of poor performing CHWPs. Both policy makers and programme managers have questions that need answering. For policy makers the key questions of this evaluation are: Has the LHWP been able to deliver simple health promotion, preventative and curative services? Have these services had an impact on health outcomes? Has the programme been able to expand access to health services and provide coverage for the poor? Have these services been provided at a low cost and are the aggregate costs affordable? For the managers of the LHWP these questions need to be complemented by further questions: 2 Ministry of Health- Government of Pakistan Prime Minister s Programme for Family Planning and Primary Health Care- Revised PC-1 August page 3. Note: The Programme was originally officially called the Prime Minister s Programme for Family Planning and Primary Health Care. This name was changed in 2000 to The National Programme of Family Planning and Primary Health Care. The programme is typically referred to as the Lady Health Worker s Programme. 3 Full-time for 3 months followed by 12 months of in-service training at one week per calendar month. vi

11 EXECUTIVE SUMMARY Where are the strengths and weaknesses of the systems supporting the HW and how can they be improved? and Bearing in mind the goals of the LHWP, are the services and treatments being provided by LHWs the most effective? The structure this evaluation is shaped around the answers to these key questions. MAIN EVALUATION CONCLUSIONS This evaluation has been able to show that that the LHWP has been able to supply a service to individuals and a large proportion of these individuals are poor. Above all the LHWP has been able to buck the international trend by providing a service that has had an impact on health, either directly (i.e. through reductions in childhood diarrhoea), or indirectly, through the uptake of preventative measures, therapies or contraceptive practices that are know to impact on health and fertility (i.e. vaccination, iron tablets for pregnant women). These impacts include: A large and positive impact on childhood vaccination rates, A large and positive impact on reversible methods of contraception (pills, condoms) especially in rural areas, and on all methods of modern contraception in rural areas, The increased uptake of antenatal services, The increased provision of iron tablets to pregnant women, Increased levels of child growth monitoring, Lower rates of childhood diarrhoea, and More positive attitudes to family planning 4. When measured both against international experience, and against a national background and culture of non-performing public services, this is an enormous achievement. The Programme is almost certainly having more of an impact per unit of cost than comparable alternative services provided in the public health system i.e. the FLCFs. The Programme has benefited from its organisational arrangements. It has: Strategic planning capability at a national level, with the opportunity to delegate operational planning for service delivery, Been able to restrict the use of arbitrary power by provincial and local politicians by ensuring adherence to selection criteria and service standards, Develop a cadre of contract workers more focused on service delivery than the average public servant, Emphasise uniformity of service and conditions provided through bureaucracy rather than patronage which has probably helped to increase the communities respect for government and health service delivery. 4 No impact on Fertility Rates (FR) and Infant Mortality Rates (IMR) was determined. Both FR and IMR are notoriously difficult to measure and notoriously slow to change. The position of the evaluation is that, whilst the LHWP may have had an impact on FR and IMR the survey instruments and available interviewing techniques were not precise enough, in this instance to measure any change. vii

12 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT However, whilst congratulations are fully justified, this evaluation has pointed to some serious issues that need to be resolved and has indicated future strategic directions for the policy and decision makers of the Programme. These issues include: The setting of unrealistic targets for the number of employed LHWs, The problems of fully funding a programme with national coverage, Quality control, The ability to continue to attract high quality managers to the most senior posts; and Continued senior government support and sponsorship for the LHWP. And for the Programme Managers: The organisational control of the LHWP Quality of service provision Low productivity/low impact LHWs Supervision support including improved supervisor mobility LHW knowledge Logistics support (medicines, contraceptives and equipment) LHW salaries paid on time. KEY POINTS FROM OTHER EVALUATION REPORTS The evaluation results are extremely rich in information with different audiences having quite different needs. For this reason a range of reports have been written (listed in the inside front cover) that can be accessed through the LHWP office in Islamabad. The following is a selection of key points related to service and costs contained in this summary and in the reports. LHWs At the end of FY 2000/01 there were 37,838 LHWs employed effectively serving 31 million people. Most communities knew and respected their LHWs and believed that the LHW had brought about improvements in health in the community. Curative Services LHWs are an important source of curative consultations for the population they serve. Almost one fifth of individuals in the served population who were ill, and who consulted any service provider, consulted the LHW. LHWs are a particularly important source of care to rural women. Half of all LHWs reported having seen an emergency case in the previous three months, suggesting that they are playing an important role in identifying and referring serious illnesses. Household reports of LHWs treatment of children with diarrhoea compare favourably with other sources of care. LHWs were the most likely of all providers to have recommended rehydration. They compare particularly well with other community-based providers such as hakeems and homeopaths, who rarely recommended rehydration. Where they are consulted, LHWs are usually playing the expected first contact role. LHWs are also acting as a link between health facilities and the communities they serve, referring about half of all cases seen to another provider, usually a government provider. viii

13 EXECUTIVE SUMMARY Preventative and Promotive Services LHWs were on average undertaking 25 household visits and seeing 20 clients in a week. Many individuals in the population served by LHWs are being provided with appropriate preventive and promotive services. Often around 40 to 50 percent of individuals have received the services they are eligible for. This represents a substantial achievement in providing primary health care services to the population. The level of provision varies by type of service. LHWs are playing an important role in the provision of family planning. Vaccination promotion reaches over two thirds of children under three, whereas only one in ten had been recently weighed by the LHW. Most services fall somewhere in between these two extremes. Service levels of LHWs are slightly higher in rural areas than urban but vary significantly between regions with service levels being higher in AJK/NA and lower in Balochistan. High vs. Low Performing LHWs LHWs can readily be divided into better and worse performing groups. Service provision by high performing LHWs is consistently high across almost all services where as poor performing LHWs consistently provide a low level of service. Higher performing LHWs have higher levels of knowledge, are better supervised and better supplied with drugs and equipment. They work longer hours than poor performing LHWs. Their supervisors also have higher levels of knowledge and are more likely to have been fully trained. The two most important factors for increasing LHW service delivery are adequate supervision particularly ensuring that supervisors have access to adequate transportation and LHWs knowledge. Supervisors access to a vehicle appears to be important even in urban areas. LHWP Costs The real unit cost (per LHW) fell by 51 percent over the seven-year period from 1994/95 to 2000/01 to around Rs.30,000. Based on recommendations to improve efficiency made by this Evaluation, the cost per LHW of a fully funded programme would be Rs.46,150. There is a clear link between financing gaps, LHW performance, LHWP impact. The very large financing gap that the LHWP has faced over most of its life suggests that the while the LHWP has managed to overcome many of the problems of the other CHWPs around the world, the Government of Pakistan is struggling to overcome the aggregate financing problem. If the programme were to meet the initial target in the R-PC1 of 100,000 fully funded LHWs, Pakistan would have to devote almost one third of the entire health budget to the programme. Expansion If the requisite resources are not made available to fully fund the LHWP, the programme should not continue to expand at the expense of providing appropriate supplies and support to the existing LHWs. ix

14

15 1. INTRODUCTION: THE LADY HEALTH WORKER PROGRAMME PRIMARY HEALTH CARE IN PAKISTAN The LHWP was conceived in 1993 and launched in April 1994 as a Federal development programme. Common to most countries at a similar stage of development, Pakistan s burden of disease is characterised by infectious diseases that are easily treatable and at low cost. These diseases (diarrhoea, respiratory tract infections, malnutrition, peri-natal infections and peri-natal complications), which disproportionately affect children, their mothers and women more generally, are currently amongst the major national causes of morbidity and mortality 5. In addition, contraceptive prevalence rates have been historically low, and fertility rates high. Since the early/mid 1990s Pakistan has grappled with alternative strategies that aim to provide the simple and low cost health and family planning services that have the potential to significantly reduce the extent and frequency of communicable diseases, as well as providing access and ensuring availability to family planning services. In the 1990s, and under the framework of the Social Action Programme, this included the Family Health Project, the Village Based Family Planning Worker programme, initiatives with clinic-based Lady Health Visitors, and also the LHWP. In providing these services a major contribution would be made to improving the health of all in Pakistan. CHALLENGES IN ESTABLISHING A COMMUNITY HEALTH WORKER PROGRAMME The strategy of establishing Community Health Worker Programmes around the world has been underpinned by some key assumptions, that: It was feasible and safe to provide preventive, promotive and curative care in communities, Services would have a significant impact on health outcomes and could be provided at low cost, The coverage of services could be extended to a large share of the population and in particular to the poor that had been underserved or unserved by health services in the past. However CHWPs have not been as successful as hoped. A review of the evaluation literature points to: Loss of focus on key strengths particularly providing low cost/high impact health interventions. 5 For recent work on Burden of Disease estimates for Pakistan see Hyder. A. et al.; Applying Burden of Disease Methods in Developing Countries: A Case Study from Pakistan, American Journal of Public Health, 90, No.8, August 2000.

16 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT Lower than expected coverage due to services reverting to being provided by clinics rather than in the community. Services having been delivered but of poor quality. This results in low demand and therefore low coverage. Services not always leading to improved health outcomes. Unaffordable aggregate costs: Very large numbers of well-supported health workers can make large claims on tight public health budgets. Furthermore, if there are rising levels of referral made by CHWPs there will be additional costs on other parts of the primary care system. Political support has been difficult to sustain to ensure adequate finance, and access to high quality management and leadership 6. Despite the fact that by the beginning of the 1990s CHWPs were almost becoming unfashionable, the Government of Pakistan took the bold step, in 1993/94 7, of launching one of the largest CHWPs in the world at that time. THE LHWP When it was designed in the early 1990s the LHWP 8 had as its objectives the public financing and provision of basic community health services to all rural villages and poor urban areas in Pakistan. The Lady Health Worker Programme (LHWP) was to be built around young LHWs that were to be resident in the communities that they served. The Revised-PC1 (R-PC1), the main planning document for the LHWP 9, provides a useful overview of the objectives and Programme implementation strategies. 10 LHWP Objectives The objectives in establishing the programme were as follows, to: Address the primary health care problems of the community, providing promotive, preventive, curative and rehabilitative services to which the entire population has effective access. Bring about community participation through creation of awareness, changing of attitudes, organisation and mobilisation of support. Improve the utilisation of health facilities by bridging the gap between the community and health services in the country through Lady Health Workers. Expand the family planning services availability in urban slums and rural areas. 6 This advocacy task is not made easy when impartial evidence of programme impact is only available periodically. 7 There were many influential voices at the time that did not support the concept of a Lady Health Worker programme. 8 The formal name of the programme was The Prime Minister s Programme for Family Planning and Primary Health Care. This name has been changed to The National Programme for Family Planning and Primary Health Care. 9 Signed by the Secretary of Health on 20 August Although the LHWP began in FY 1993/94 it was substantially revised in 1995 to incorporate the following main changes: The extension of the programme to urban as well as rural areas; the upward revision of the recruitment target to 100,000 Lady Health Workers by 1998; the addition of a cadre of LHW Supervisors to guide the implementation of the programme; a Health Management Information Systems (HMIS) component, and; a Mobile Health Education Campaign (MHEC) component. 2

17 INTRODUCTION: EVALUATING THE LADY HEALTH WORKER PROGRAMME Gradually integrate existing health care delivery programmes like EPI, Malaria control, Nutrition, MCH within the Prime Ministers Programme. 11 These objectives were to be achieved by providing a set of core services and achieving a set of ambitious targets (described in detail by the RPC-1). Targets were set for; the numbers of LHW to be recruited; numbers of health officials to be trained as trainers; number of Lady Health Supervisors (LHS); amount and range of medicines and contraceptives to be purchased; and number of vehicles to be bought. At its inception the LHWP aimed to have 100,000 LHWs in post, covering 100,000 communities and 100 million individuals. The target programme size was to be reached by 1998 at a cost of Rs.9.1 billion. The target number of LHWs remains the key indicator for the programme overshadowing all other indicators. LHWP Critical Design Factors In its design and implementation the programme designers took account of experience from both national and international community health worker programmes. It was accepted by the government that for the LHWP to be effective, investment was needed in: Substantive training for both Lady Health Workers and their supervisors, Incentive systems to ensure high standards of productivity, 12 Supervision and inspection systems that offered both learning opportunities and incentives for performance, Logistics systems that ensured constant supplies of medicines and family planning materials to LHWs, Budgeting and expenditure control systems, and Political capital to assure continued budgetary support and access to highly committed and competent leadership and management. Planning Periods and Horizons Since the R-PC1 was approved, the programme implementation period has twice been extended to 2001 and then to 2003 when the programme currently expects to recruit its 100,000 th LHW. Despite the extension, the LHWP s basic cost structure has not been altered substantially and the R-PC1 remains the programme s main planning document, and it is this document that is used as the basis for annual programme planning. Management Structure The programme was designed so as to have a light management super-structure and to be implemented by units attached to the Federal Ministry of Health (MoH) (Figure 1-1). These units include the Federal Programme Implementation Unit (FPIU) attached to the Federal MoH; the Provincial Programme Implementation Units (PPIUs) attached to the Provincial Departments of Health (DoH); and the District Programme Implementation Unit (DPIU) that is attached to the District Health Office (DHO). These Implementation Units are staffed either by public servants on deputation or contract employees. 11 Ministry of Health- Government of Pakistan Prime Minister s Programme for Family Planning and Primary Health Care- Revised PC-1 August page Combining salaries, incentive payments, and the use of short-term contracts. 3

18 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT Whilst the LWHP is embedded in both the Ministry and Departments of Health, and whilst budget and expenditures are controlled by Ministry staff on deputation, the LHWP still has accounting and information systems that are separate from the main MoH and DoH systems. Initially, it was planned for the staff of the First Level Care Facilities (FLCF) to train and supervise the LHWs. Eighteen months after the programme began it was decided to develop a separate cadre of female contract employees, the Lady Health Supervisors (LHSs), to be responsible for ongoing supervision and monitoring. The Figure 1-1 Management structure of the LHWP LHS reports to and attends a monthly meeting at the DPIU. FPIU Federal MoH Field Programme Officers (FPOs) PPIU Provincial DoH employed by the PPIU inspect 2-3 districts each reporting on their findings to the PPIU. DPIU Districk District Health Office LHS/FLCF FLCF The LHW works from her home, where she is required LHW Clients to have one room designated a Health House. Decision-Making Responsibilities The RPC-1 provides the parameters for decision-making within the Programme. Essentially, the most important decision, in terms of policy formulation and budgeting, revolves around the number of LHWs employed. It is the Secretary of Health, in conjunction with the Planning and Development Division, who takes decisions on LHW numbers. Decisions on policy matters within the RPC-1 guidelines or operational initiatives that do not incur expenditure are taken within the FPIU. While most of the policy level decision-making power is held at the Federal level, most operational decisions are the responsibility by the Provincial and District Implementation Units. For example: Decisions on the allocation of LHW within the district are taken by the District with the approval of the Provinces, Decisions of the allocation of LHWs to districts are taken at the provincial level, and Staff appointments for the hiring and firing of LHWs, LHSs and their drivers are undertaken at the district level. In its current structure, therefore, the LHWP is centrally funded and directed from the Federal level, but key operational decisions are taken at the provincial and district levels. Finally, while the National Coordinator of the FPIU holds the budget for the programme he does not have direct authority over the Provincial Coordinators, who remain employees of the Provincial Secretary of Health. However, the programme design aims for 4

19 INTRODUCTION: EVALUATING THE LADY HEALTH WORKER PROGRAMME close coordination and cooperation between the managers of the FPIU and the PPIUs with regular meetings between the two units. 5

20

21 2. ISSUES AND SCOPE OF THE EVALUATION This evaluation, commissioned by the LHWP itself, is essentially about whether the LHWP has been able to buck both the international and national trend of poor performing CHWPs. Both policy makers and programme managers have questions that need answering. For policy makers the key questions of this evaluation are: Has the LHWP been able to deliver simple health promotion, preventative and curative services? Have these services had an impact on health outcomes? Has the programme been able to expand access to health services and provide coverage for the poor? Have these services been provided at a low cost and are the aggregate costs affordable? For the managers of the LHWP these questions are complemented by further questions: Where are the strengths and weaknesses of the systems supporting the LHW and how can they be improved?, and Bearing in mind the goals of the LHWP, are the services and treatments being provided by LHWs the most effective? THE STRUCTURE OF THIS EVALUATION The structure of this evaluation is shaped around the answers to these key questions. The resources available to this project have allowed a comprehensive evaluation. We have used a wide range of research instruments 13 in order to investigate the performance of the LHWP. In addition to research, the evaluation paid considerable attention to the dissemination and discussion of the evaluation s results. 14 As a result, the final products 15 of the evaluation not only reflect the broad scope of the evaluation, but also the processes that have been followed to ensure that decision makers needs for analysis and information were addressed, and that the results of the evaluation are communicated and discussed with those who will be responsible for developing the programme over the next few years. Given the wealth and detail of documentation this summary paper is focussed on providing a reference to the key results, findings and judgements of the evaluation and the future challenges that are covered in detail in one or more of the 12 reports. 13 These include five quantitative surveys that provide estimates of performance for each province, AJK and FANA (see Quantitative report for detailed results and the Survey Sampling Design Report for sampling design); key informant interviews with more than 60 senior programme staff, Ministry of Health officials and national experts and observers of public health care in Pakistan; participatory and qualitative research in four districts; two major reviews of programme expenditure and financial performance; a systems study in three districts; and a review of training appropriateness. 14 Printed materials are available from the LHW Programme offices in Islamabad. 15 Twelve reports that have been produced as part of a package of final products which are listed inside the front cover of this report. In addition, during the life of the evaluation (which began in March 1999 but was suspended soon afterwards for one year after the change in government) there were 24 dissemination workshops; three major interim reports; 11 evaluation newsletters distributed to all districts; and numerous briefing notes. A large number of private briefings were undertaken with senior Lady Health Worker Programme (LHWP) and Health Ministry staff..

22 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT This paper is structured to answer the following questions: Section Three: Has the LHWP been able to deliver simple health promotion, preventative and curative services? And has she delivered them to the poor? Section Four: answers the question: where are the strengths and weaknesses of the systems supporting the LHW and how can they be improved? Section Five: Whether these LHW services have had an impact on health outcomes? Section Six: Have the services been provided at low cost, and in aggregate, can they be afforded? Section Seven: What are the key judgements and issues of the evaluation? EVALUATION APPROACH Given these aims and this approach to improving a multiple range of health outcomes, how can an assessment of performance be structured? In the case of less complex interventions, the standard approach is to employ a cost-effectiveness analysis. 16 While a rigorous costeffectiveness analysis is not feasible for a range of technical and practical reasons the cost effectiveness framework is straightforward. 17 The impact of the LHWP across the country will depend upon: The service coverage in those communities served by CHWs (in particular, the proportion of eligible clients seen by the CHW (Section 3), The quality of the inputs provided (especially the appropriateness of training, supplies and equipment)(section 3), The chosen package of interventions in which LHWs are trained and supported (especially their clinical efficacy, and the balance between promotion, prevention and curative care)(section 3 and Section 5), The poverty focus that is achieved (Section 3). These components of the care provided can then be judged according to their: impact on health status (Section 5), and costs as compared with comparable alternatives (Section 6). It can be seen that poor performance in any one component can have an impact on final health improvements. That said, failure or poor performance in one part of the system should not necessarily result in the condemnation of the whole system. The judgement to be made is rather whether improvements in weak parts of the system can be made and will result in improved health status, and at what cost. 16 That is an analysis of the cost of achieving a defined improvement in health status. 17 Berman. Peter A., Community Health Workers: Head Start or False Start Towards Health for All? Social Science and Medicine, 25, Health Development Network, The World Bank, Washington D.C. May

23 3. THE SERVICES PROVIDED BY THE LHW INTRODUCTION In general, the international evidence suggests that CHW programmes significantly increase the volume or levels of services provided, and cover households and individuals that were otherwise not served or poorly served by health services. However, the evidence also suggests that CHW programmes can buckle under the pressure of demand for curative care and shift their service packages away from what is needed (preventative care) to what is wanted (curative care). 18 What services does the LHW provide and who does she provide them to? This section provides the evaluation results on the: The technical mix of services Population coverage of the LHWP, Preventative, promotive and curative services that are provided by the LHW, Quality of service that is provided. The poverty focus of the LHWP in coverage and services THE TECHNICAL MIX OF SERVICES The mix of health services provided by the LHW is likely to be important in determining impact. The LHWP has shown itself diligent in its continuous high level clinical reviews, both of any changes in the patterns of disease and of any changes in the patterns of service that should be provided by LHWs. High level reviews of the mix of services are complemented by local level flexibility. Local level flexibility allows DPIUs and PPIUs to respond to local needs, and local differences in capabilities to provide services in addition to the standard national level package e.g. in Sindh some LHWs have been used to directly provide vaccination as part of an EPI team. POPULATION COVERAGE The total number of individuals served provides a basic measure of Programme coverage. In order to estimate the number of individuals that are served and visited by the LHWP the evaluation has estimated the number of: LHWs currently employed, Households and individuals registered and served by LHWs, Households visited by LHWs. 18 Onwukwe. Iheadi A., Mark Pearson Assessing the Effectiveness of Community Health Workers, (mimeo), Institute for Health Sector Development, July 2000.

24 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT The Number of LHWs Currently in Employment Establishing the actual numbers of LHWs is critical both for understanding the aggregate costs of the LHWP and also the national level impact. 19 Three different data sources were used to estimate the number of LHWs currently in employment: The LHWP database The Evaluation s Sample Survey The Evaluation s count up of the payroll. In summary, the evaluation concluded that the LHWP database is out of date and as such it is not a reliable source of data for estimates of LHW numbers. Of the three data sources the payroll is likely to yield the most accurate estimate of LHW numbers. From this source it is estimated that at the end of FY 2000/01 the total numbers of LHWs actually employed in the LHWP was 37,838. The Numbers Registered and Served by LHWs Using the Evaluation s sample survey it was possible to estimate both the number of households and the number of individuals registered by the LHWP. LHWs are supposed to serve a population of 1,000 individuals, or approximately 200 households. In practice, each LHW on average has 145 households registered however, as average household size is quite high (7.1 individuals) the average number of individuals registered with each LHW is 980, very close to the Programme norm. While registered households were not generally found to be fictitious, some 17 percent of the interviewed households did not know that they were registered by the LHW. Therefore, given the numbers of LHWs employed this suggests that by FY2000/01, across the country approximately 37 million individuals were registered and that almost 31 million were effectively registered. Household Visits Made and Clients Seen The programme norm is for the LHW to visit each of her households monthly. LHWs were asked how many household visits they undertook in the week preceding the survey. The mean number of visits reported was 25 (Table 3.1). At that rate, if LHWs were visiting all of their households, all the registered households should be visited, on average, in around one month and a half. From the household interviews we found that slightly under three quarters of households said that they had been visited by the LHW in the preceding three months (72 percent). 20 While this is a positive finding it does there remain a substantial number of households that are not receiving regular visits, despite being registered with the LHW. 19 There is considerable discussion of this topic in both the Quantitative Survey Report (Chapter 3) and in the Financial and Economic Analysis (Chapter 3). 20 When reporting about the preceding week, households report a higher level of household visits than do the LHWs themselves. However, these levels are too high to be plausible it seems most unlikely that almost half of the LHWs households were visited in the week preceding the survey and it may reflect respondents protecting the LHW in their answers. 10

25 THE SERVICES PROVIDED BY THE LHW Table 3.1 Number of household visits made during the preceding week Measure Value Number of household visits: (%) Up to More than 50 5 Total 100 Mean number of household visits made 25 The LHWs were also asked how many patients/clients they saw in the week preceding the survey. This specifically included individuals to whom they only gave advice. On average, LHWs reported seeing 20 individuals. Once again, there is a wide variation in the number of clients seen. More than one third of LHWs reported seeing ten or fewer clients. While there are no explicit targets for the number of clients that a LHW should see in a week, this clearly represents under-utilisation of the service. The community survey 21 conducted as part of the evaluation showed that most communities knew their LHW, knew that she was working as an LHW and believed that women are respected as LHWs. Most said that it was usually possible to go the LHW s house for a consultation. Almost three-quarters also believed that the LHW had brought about improvements in health in the community. The improvements cited most frequently were improvements in health status, increased health education and increased visits from the vaccinator. The community reports did suggest, however, that around 20 to 30 percent of LHWs are not regularly undertaking visits to households or married women. PREVENTATIVE AND PROMOTIONAL SERVICES Four important client groups are identified in Table 3.2. The client groups are: the household itself (as a unit); women who have had a birth since 1997; currently married women aged 15-49; and children under 3 years of age. LHWs provide a range of promotive and preventive services to these groups. The table shows the extent to which LHWs: provide hygiene education on drinking water and sanitation, provide nutritional advice and growth monitoring, monitor and advise women on their health, and that of their babies, after birth, supply and refer women for family planning, motivate and educate women on family planning, and promote and facilitate vaccination. 21 Two groups of respondents were gathered together for the community interviews one of men and one of women. 11

26 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT Table 3.2 Preventive and promotive service delivery by LHWs Measure Value Households registered with the LHW: % who report that the LHW has ever talked to them about ways to improve: the cleanliness of drinking water 45 hygiene and reduce diarrhoea 44 Women who had a birth since 1997 (reporting on their last birth): % who report that the LHW: Gave them advice on which foods to eat while pregnant 45 came to see her and the baby within 24 hours 10 came to see her and the baby within 7 days 38 Saw the baby within 7 days and weighed the baby (1) 14 Saw the baby within 7 days and gave advice on breastfeeding (1) 28 Gave her advice on family planning within 3 months of the birth 40 Currently married women (aged 15-49): % of current users of modern contraceptives who were supplied by the LHW 20 % of current users of modern contraceptives who were supplied or referred by the LHW 32 % of current users of pills and condoms who were last supplied by the LHW 44 % of non-users of modern contraceptives who have ever discussed family planning with the LHW 41 % of non-users of modern contraceptives who have discussed family planning with the LHW within the last 6 months 26 % of current users of modern contraceptive, who were not supplied or referred by the LHW, who have ever discussed family planning with her 44 % of current users of modern contraceptive, who were not supplied or referred by the LHW, who have discussed family planning with her in last 6 months 27 Children under age 3 years: % who have a health card provided by the LHW 9 % whose mothers say that the LHW talked to her about vaccinating the child 67 % whose mothers say that the LHW encouraged her to take the child for vaccination at the age when it was necessary 60 % whose mothers say that the LHW gave her advice on feeding the child 41 % ever weighed by the LHW 27 % weighed by the LHW in the previous 3 months 11 Note: (1) Denominator is all eligible births, LHWs present at birth (4% of cases) are not included in numerator. The level of provision varies by type of service. Vaccination promotion reaches over two thirds of children under three, whereas only one in ten had been recently weighed by the LHW. Most services fall somewhere in between these two extremes. LHWs are playing an important role in the provision of family planning. Levels of service provision are similar in urban and rural areas. Rural areas show levels of provision at least as high as urban areas. LHWs are a more important source of family planning in rural areas than in urban areas. However, even in urban areas, where there is a wider range of alternative sources, LHWs continue to supply an appreciable fraction of users in the populations that they serve. Differences in service provision by stratum are much larger and show a reasonably consistent pattern. Service delivery levels are highest in the stratum that combines AJK and FANA, almost without exception. The differences from other strata are substantial; in many cases more than half of eligible clients are provided with services. Balochistan has the lowest level of service provision for most of the measures shown. The main exception to this is in 12

27 THE SERVICES PROVIDED BY THE LHW the supply of contraceptives, where LHWs share of provision is high. This presumably reflects a lack of other sources of contraceptives. It shows that LHWs in Balochistan are playing an important role in this area despite underperformance in other services. CURATIVE SERVICES While there is little research showing positive links between improved health outcomes and prevention and promotion activities, there is more evidence for the impact on health outcomes of the simple treatments of the type offered by LHWs. Of the sick or injured people who consult a health practitioner, one fifth will consult their LHW. (Table 3.3). Similar levels of consultation were reported for children under five who suffered from a respiratory infection; in the case of diarrhoea, the level of consultation was slightly lower but still substantial. Given that there are a number of other sources of care available, this level of use indicates some confidence in the LHW on behalf of the households served. As would be expected, females are more likely than males to consult with the LHWs. LHWs role in curative care is substantially larger in rural areas than in urban areas (Table 3.3). This is particularly true for rural women and girls; over one quarter of females who had been ill consulted a LHW, if they consulted any care provider. Perhaps surprisingly, urban / rural differences are not so pronounced in the case of childhood diarrhoea and respiratory infections. Table 3.3 Consultations with the LHW by sick individuals by place of residence Measure Urban Rural Total Individuals who were ill or injured in the previous fourteen days: % who consulted the LHW total % who consulted the LHW female % who consulted the LHW male Children under 5 who were ill in the previous fourteen days and who consulted any health provider: % with diarrhoea who consulted the LHW % with respiratory infection who consulted the LHW The extent to which LHWs are consulted also varies between provinces. In AJK/FANA, almost one third of individuals who were unwell, and who consulted anyone, saw the LHW. Sindh has the lowest consultation rate. AJK/FANA also had the highest consultation rate for children under five who had diarrhoea. Balochistan had the lowest - only one percent of cases consulted the LHW there. The main reason why LHWs were not consulted for children under five, at least was that the LHW was not available or was not helpful (Table 3.4). 22 This is a problem that improved supervision and training ought to be able to address. When the lack of medicines is added to this, almost half of the reasons given for not taking up the service are due to factors 22 Reasons given for not consulting the LHW when the child had a respiratory infection were similar. 13

28 LADY HEALTH WORKER EVALUATION SUMMARY OF FINAL REPORT that the Programme should be able to improve and which will improve the impact on health status. Table 3.4 Reason for not consulting the LHW: children under five with diarrhoea Reason Percentage LHW not available/not helpful 37 LHW cannot treat diarrhoea 3 Lack/poor quality of medicines 10 Preferred consultation elsewhere 17 Consultation was not necessary 12 Other 21 Half of all LHWs reported having seen an emergency case in the previous three months, suggesting that they are playing an important role in identifying and referring serious illnesses. Where they are consulted, LHWs are usually playing the expected first contact role. LHWs are also acting as a link between health facilities and the communities they serve, referring about half of all cases seen to another provider, usually a government provider. LHW s Treatment and Referral Practices Respondents in the household interviews were asked about the treatment and advice given to children with diarrhoea. The results suggest that LHWs show relatively good practice in the treatment of simple diarrhoea. Of all care providers, LHWs are the most likely to recommend oral rehydration solutions (both packet and home made) and an increase in the provision of other liquids. Together with private providers, they are the most likely to advocate continued feeding and breastfeeding (recommendations which would be applicable only to a fraction of the children). There was no clinical evaluation of cases and there may have been differences in the type or severity of the cases seen by the different care providers. Nevertheless, the results are encouraging and the contrast with other community-based providers is particularly striking: less than one in twenty of the hakeems and homeopaths were reported to recommend any form of rehydration. LHWs are expected to treat many cases themselves; cases that they cannot treat are referred to health facilities. This is intended to channel serious cases to the facilities, while reducing the burden imposed on facilities by simple cases that can be treated in the community. Information given by the LHWs about the last case seen suggest that half of all clients they see are referred (49 percent). 23 Half of the referred cases are sent to the LHW s FLCF (Table 3.5). Almost half of the remainder are referred to another government health facility and the other half to a private source of health care. 23 Without an independent clinical assessment of the case, it is not possible to evaluate whether these referrals are appropriate. 14