TERMS OF REFERENCE FOR THIRD PARTY EVALUATION A. BACKGROUND OF CONTRACTING IN COUNTRY X Challenges: The Government of Country X (GOX) has committed itself to dramatically improving the health and well-being among the poor in Country X. Achieving this improvement will require, among other things, a strengthened primary health care system that can deliver high impact preventive, curative, and promotive services. These services, if of sufficient quality and reach, will help reduce child and maternal mortality, contain the spread of infectious diseases, and reduce other threats to good health. Currently, the primary health care system faces a number of important challenges including: (a) low utilization of Primary Health Care Centers (PHCCs) for curative and promotive services; (b) low coverage of preventive services; (c) inconsistent and often poor quality of care; (d) frequent absenteeism of staff and a shortage of staff in remote areas; and (e) inequitable coverage of services so that the poor and those living in remote rural areas have limited access. Basic Approach: In response to these challenges, the GOX is interested in undertaking public-private partnerships (PPPs). The current proposal for a PPP envisages an area/community-based approach for delivering primary health care services in PHCCs and their surrounding catchment areas. The basic approach for PPP is: (a) the private sector partner of the GOX (hereafter referred to as the Partner ) is accountable for achieving tangible results (described below) in delivering a package of primary health care services; (b) in order for the Partner to achieve the expected results, the GOX will provide it adequate and timely financial resources; (c) careful monitoring and evaluation will be carried out so that the GOX can be confident it is getting value for money; (d) both parties will focus on outputs and outcomes more than inputs and processes which means that the GOX will specify what results should be achieved and services delivered but that the Partner will be given latitude on how to implement those services; (e) the Partner will be provided with sufficient managerial autonomy so it can flexibly respond to local conditions and introduce innovations aimed at improving service delivery; and (f) an explicit agreement will facilitate the relationship between the GOX and the Partner by making roles and responsibilities explicit. Pilot Test of PPP: The GOX is interested in evaluating whether this type of PPP can achieve the following objectives: (a) Significantly strengthen primary health care in PHCCs and their associated catchment areas so as to ensure the wide-spread delivery of a standard package of preventive, curative, and promotive services that will help improve the health and well being of Country X.
(b) Dramatically improve the: (i) coverage and utilization of services; (ii) quality of care; and (iii) equity of access to services by geographical areas, by income level, and by women and children. (c) Ensure that patients and communities are increasingly involved and satisfied with the publicly financed health services and facilitate the community s participation in the design, delivery, and evaluation of health services. A third party firm or organization is required to design, conduct, and analyze the baseline and follow-on studies that will evaluate the PPP pilot test. B. OBJECTIVES AND DESIGN OF THE PPP EFFORT Indicators of Success: Achievement of the above objectives will be assessed by the indicators and targets described in Table 1. By the end of the four year period covered by PPP, it is expected that the significant progress will have been made towards the targets in Table 1. These targets may be revised as data become available and are meant to be indicative rather than exact. What will matter is significant progress along these parameters. Table 1: Key Performance Indicators and Targets Indicator Baseline 1 Approximate Target 1. Number of consultations per person per year provided by the PHC and its outreach activities 2. Contraceptive Prevalence Rate - % of couples of reproductive age currently using a modern family planning method 3. TB case detection rate (number of sputum positive cases detected as % of target based on estimated prevalence, i.e., case-finding.) 4. Proportion of children 6 to 59 months that have received Vitamin A supplement within last 6 months. 5.Vaccination: measles immunization coverage before 12 months of age among children 12-23 months of age. 6. Coverage of antenatal care -% of all pregnant women receiving at least one antenatal care visit from a skilled provider. 7. Proportion of births attended by skilled attendants (includes institutional delivery but excludes trained TBAs) 8. Score out of 100 on an index of quality of care as judged by 3 rd party, which includes the adequacy of waste management. 9. Improved equity ratio of poorest to richest income quintiles (based on asset index) on number of consultations. Means of measuring indicator 0.3 1.0 HMIS & HHS 9% 15% HHS 26% 45% HFA & HMIS 36% 55% HHS & HMIS 28% 50% HHS & HMIS 22% 50% HHS & HMIS 14% 25% HHS & HMIS 46 70 HFA 0.42 0.60 HHS
1 Baseline data is approximate and will have to be updated. NOTE: HHS=household survey, HMIS=health management information system, HFA=health facility assessment. Package of Services: The PPP aims to strengthen the delivery of a package of standard primary health care services, including: (a) maternal and child immunization; (b) antenatal, obstetrical, and post-partum care; (c) family planning services including all modern methods (injectables, IUCD, condoms, and pills) and including family planning operations (NSV and tubectomy); (d) diagnosis and treatment of major infectious diseases including tuberculosis, malaria, and kala azar, etc.; (e) basic curative services normally available PHCCs; (f) nutritional support including improving micronutrient deficiencies (vitamin A, iron, iodine), therapeutic feeding, and breast-feeding promotion; (g) participating in special health activities such as national immunization days and other types of campaigns; and (h) carrying out public health functions such as disease surveillance and recognition/response to epidemics. Size of Each PPP Lot: In the three districts of Province Y and four districts of Province Z, and three districts of Province W (a total of 10 lots ), all the 105 PHCCs and their catchment areas will be involved in the PPP scheme. Each district will constitute one lot and has an average population of 450,000 (the smallest has a population of 380,000 and the largest 520,000). Evaluation Methodologies: The achievement of the indicators in table 1 and certain other aspects of health care (such as community satisfaction, health expenditures, use of unqualified providers) will be assessed using the following methodologies: (a) (b) (c) Household surveys: In each of the 10 PPP lots, and 10 comparison areas, household surveys will be carried out. The baseline survey will be done as soon as possible, with a mid-term survey in July 2011, and the end-line survey in June 2014. Health facility assessments: An in-depth health facility assessment will be carried out annually among a random sample of health facilities in PPP districts and comparison areas. The assessment will examine the quality of care, broadly defined, and other related indicators such as staff morale. Health Management Information System (HMIS): HMIS data will be used by the Provincial Health Departments to monitor, on a monthly basis, the performance of the PPP and the comparison PHCCs. C. OBJECTIVES OF THE CONSULTANCY This consultancy aims to: (a) design, conduct, and analyze the different types of studies needed for judging the effectiveness of PPP in improving health services in Country X; (b) design the overall evaluation such that it provides the strongest evidence for decision makers; and (c) build the capacity of the MOH to conduct, analyze, and use such information.
D. SCOPE OF SERVICES The selected firm/organization will work closely with the GOX and under the direction of the GOX s task manager to carry out the following: (a) Carefully discuss the PPP pilot with the GOX staff and provide an evaluation design that provides the information necessary to judge the effectiveness of the PPP intervention; (b) Develop household survey instruments that build upon international best practice in measuring utilization of health services, health outcomes, satisfaction with care, cost to consumers, and quality of care; (c) Also building on international best-practice, develop the instruments and methods for a health facility assessment that examines quality of care broadly defined, and other related indicators such as patient satisfaction and staff morale. (d) In consultation with the GOX determine the sample sizes required to attain reasonable statistical power; (e) Ensure the household and health facility instruments are discussed with the GOX and subjected to peer review, after which they will be field tested and modified accordingly; (f) Maintain the schedule agreed during negotiations for carrying out the baseline surveys expeditiously; (g) Ensure that surveyors are properly trained and supervised. Design and implement a quality assurance mechanism for data collection and entry acceptable to the GOX; (h) Enter and clean the data using best practice techniques and prepare the data for analysis, including getting it in a state where it could be made publicly available on a website; (i) Analyze the baseline data and prepare a concise and user-friendly report for the GOX and other stakeholders, including the private sector partners. Carry out such additional analysis as the stakeholders reasonably request to make use of the data; (j) Repeat the household survey and conduct a health facility survey 18-24 months after the baseline evaluation using the same methodology and instruments. While questions may be added to address key issues that arise during the study, only in unusual circumstances would questions in the baseline surveys be dropped; (k) Analyze the end-line data and prepare a concise and user-friendly report evaluating the whole experiment for the GOX, the Client, and other stakeholders, including the private sector partners. Carry out such additional analysis as the stakeholders reasonably request to make use of the data;
(l) Develop an explicit plan, acceptable to GOX, for building the capacity of GOX staff and decision-makers in the design, conduct and analysis of household and health facility surveys; (m) Implement the capacity building plan with due attention to quality and timeliness; (n) Present the data during meetings and workshops set up by the GOX to disseminate the results of the study. Prepare the results for publication in the international health literature; and (o) Carry out such tasks as the GOX s task manager reasonably requests to ensure efficient, high quality, and expeditious design, collection, analysis, and dissemination of the data. E. LOCATION AND DURATION OF SERVICES The successful firm/organization will need to maintain a presence in Country X during the baseline and end line evaluations. The exact districts have yet to be determined. It is expected that the consultancy will have a total duration of about 26 months. However, limited work will be needed between baseline and end line assessments. F. QUALIFICATIONS AND EXPERIENCE The successful firm/organization will have: (a) (b) (c) (d) (e) (f) (g) a minimum of five years of experience in evaluating health or other social services with extensive experience in developing countries; a successful track record of carrying out high quality evaluations in developing countries; staff who have published articles in peer-reviewed journals on evaluation of health services; staff with facility in English, both written and oral; a demonstrated ability to get along easily with governments and NGOs and deal successfully with disagreements; a facility with computers and a variety of software; and knowledge of Country X would be an advantage. G. SERVICES AND FACILITIES PROVIDED BY THE CLIENT The client will: (a) (b) provide the firm/organization relevant information related to the consultancy, such as previous similar evaluations, existing survey instruments, etc; and ensure that its staff are available for periodic meetings as needed;
H. CONSULTANT S RESPONSIBILITIES It is the responsibility of the consultant to have their own computers. The consultant will also be expected to provide electronic and hard copies of all materials developed during the consulting assignment. Electronic files should be presented in common use software.