Afghanistan Health Sector Balanced scorecard A TOOLKIT TO CALUTATE THE INDICATORS

Transcription

1 Ministry of Public Health, Afghanistan General Directorate of Policy and Planning (GDPP) Afghanistan Health Sector Balanced scorecard A TOOLKIT TO CALUTATE THE INDICATORS Johns Hopkins University, Bloomberg School of Public Health, Baltimore, USA Indian Institute of Health Management Research (IIHMR), Jaipur, India

2 Contents SECTION 1: INTRODUCTION Overview Why a detailed toolkit? The Framework: What is the Balanced Scorecard?... 2 SECTION 2: THE PROCESS Development of the scorecard Stakeholder Involvement Adaptation of the Balanced Scorecard Process of development of indicators Balanced Scorecard Domains and Indicators Collection of Data Data Analysis Disseminating the scorecard SECTION 3: DETAILED DESCRIPTION OF DOMAINS AND INDICATORS Domain A: Patients and Community Patient Satisfaction Patient Perceptions of Quality Index Written shura-e-sehie activities in the community Health worker Satisfaction Index Salary Payment Current Equipment functionality index Drug availability index Family planning availability index Laboratory functionality index Staffing index meeting minimum staff requirements Provider knowledge score Staff received training in last year HMIS Use Index Clinical Guidelines Index Infrastructure Index Patient Record Index Facilities having TB register Patient History and Physical Exam Index Patient Counseling Index Proper Sharps Disposal Average new outpatient visit per month (BHC) Time spent with patient BPHS facilities providing antenatal care Delivery care according to BPHS Females as % of new outpatients Page # 1

3 SECTION 1: INTRODUCTION 1.1 Overview This toolkit outlines the process of implementing the Balanced Score Card approach for use as a monitoring and evaluation framework of health system in the Islamic Republic of Afghanistan at the national and provincial level. 1.2 Why a detailed toolkit? This document provides detailed information on how the BSC was developed and implemented in Afghanistan. All the processes and steps described in this document are based on the experience of developing and implementing the Afghanistan National Health Services Performance Assessment conducted annually between the years 2004 and The main aim of the assessment is to monitor and evaluate the Basic Package of Health Services implemented by the Ministry of Public Health, Afghanistan (MoPH) with the active help of its development partners. This toolkit explains: What the conceptual framework of the BSC is, Who the primary stakeholders are, How the indicators were selected, How data required for calculating the indicators were collected and analyzed and How the information was presented to the Ministry of Public Health and its partners. Anyone interested in replicating the Balanced Scorecard in a different environment may find this toolkit to be a useful resource. 1.3 The Framework: What is the Balanced Scorecard? The BSC was first developed in the early 1990's by Kaplan and Norton of the Harvard Business School as a strategic management tool. The Balanced Scorecard is a management framework that enables organizations to clarify their vision and strategy and translate them into action. In the past ten years, numerous Health Service Organizations have used the BSC in much the same manner as organizations in other sectors. Afghanistan s application of the BSC at the level of a national health system is a first for a developing country. The purpose of the Afghanistan Health Sector Balanced Scorecard is to summarize the performance of the provinces of Afghanistan in delivery of the Basic Package of Health Services (BPHS), and to identify areas of strength and weakness. The Balanced Scorecard provides a framework to look in an efficient manner at multiple areas of the health sector called domains. This allows the Ministry of Public Health and other stakeholders in the health sector to see how provinces and the country as a whole are performing in provision of health services. Since the BSC is updated each year in Afghanistan, scores from one round of the BSC can be compared to scores from previous rounds to identify areas in which progress has been made and areas in which no progress has been made. The BSC is not just a measurement tool; it is used by the MOPH to clarify its vision and strategies, and to manage change. The BSC provides a framework to organize activities and learn from experience. Page # 2

4 Page # 3

5 SECTION 2: THE PROCESS 2.1 Development of the scorecard The Johns Hopkins Bloomberg School of Public Health and the Indian Institute of Health Management Research (JHU/IIHMR) were contracted by the Ministry of Public Health through a grant from the World Bank to provide technical assistance in assessing performance in the delivery of the Basic Package of Health Services (BPHS) in Afghanistan. The JHU/IIHMR Third Party Evaluation Team was contracted to conduct three rounds of the National Health Services Performance Assessment and two mid round assessments that cover a smaller number of provinces and indicators. The first round of the National Health Services Performance Assessment was conducted from August October Since 2004, NHSPA has been conducted on an annual basis during the summer months (July October). A Balanced Scorecard was adopted by MOPH and the JHU/IIHMR Third Party Evaluation team as a monitoring and evaluation framework for BPHS implementation. In April 2004 the MOPH established a Monitoring and Evaluation Advisory Board. One of the mandates of the Advisory Board was to oversee the development of the Afghanistan Health Sector Balanced Scorecard. 2.2 Stakeholder Involvement The JHU/IIHMR Third Party Evaluation Team, along with partners at the Ministry of Public Health, gave a series of presentations on the BSC as a strategic tool and demonstrated its usefulness by giving examples from health and other sectors. Discussion was initiated regarding possible domains to be used for the Afghanistan health sector. The Afghanistan Health Sector Balanced Scorecard was designed through a series of workshops and discussions with the MOPH, NGOs, and other development partners. The JHU/IIHMR team served as facilitators and technical advisors. The original two workshops involved participants working as individuals, small groups, and in plenary to arrive at a consensus on: 1) The purpose of the BSC in Afghanistan s health sector 2) The meaning and selection of the domains to be used in the BSC 2.3 Adaptation of the Balanced Scorecard In order to adapt the BSC for use at the level of a national health system, some changes to the design of the BSC were required. In the Afghanistan Health Sector BSC, customers or primary stakeholders have to come first, and mission results are measured through the perspective of the needs of clients and communities. The financial perspective, which in many versions of the BSC reflects the financial interest of the enterprise using the BSC, was shifted away from profits to transparency for clients and protection of poor patients. The domain measuring internal business process was divided into two domains one measuring the capacity of the system to provide services and the other measuring service outputs and the quality of the services being provided. After the completion of the stakeholder workshops, the M&E Advisory Board selected six domains for the Afghanistan Health Sector Balanced Scorecard. The BSC domains summarize the health sector from the following six perspectives (Figure 1): Page # 4

6 1. Patients and Community 2. Staff 3. Capacity for Service Provision 4. Service Provision 5. Financial Systems 6. Overall Vision Figure 1: Framework for the Afghanistan Health Sector Balanced Scorecard 2.4 Process of development of indicators In order to finalize the indicators that would go into each domain, two workshops were organized with stakeholders. One workshop was attended by MOPH leadership, program managers and technical staff. Another workshop was attended by staff from national and international NGOs and other institutions. A presentation of the scorecard and possible domains was made and workshop participants were requested to prioritize indicators. Participants deliberated on the preliminary indicators and their prioritization. It was agreed that approximately 30 indicators would be selected across the six Page # 5

7 domains. Participants split into groups corresponding to each domain and each group prioritized indicators within their assigned domain. Each of the proposed indicators was then analyzed by the Monitoring and Evaluation Advisory Board according to the following criteria: 1) reliability; 2) completeness (i.e. extent of missing values); 3) outlying values; and 4) sufficiency of variation. Indicators comprising a domain were then assessed as a group to ensure that there was a good balance of inputs, processes and outputs. Based on this analysis, a revised list of indicators was proposed for each domain, and a second round of workshops conducted to establish agreement on a reduced set of indicators and proposed target levels for the domains. A final BSC with 28 indicators spread across the six domains was approved by the Advisory Board in late April The M&E Advisory Board later split one indicator, the Patient Provider Care Index, into two separate indicators, based on statistical and theoretical considerations. 2.5 Balanced Scorecard Domains and Indicators Domain A: Patient & Community Results This domain captures important issues related to clients of health services and other community members. It includes patient satisfaction, patients perceptions of the level of quality of available services and levels of community involvement in the health system. The indicator of community involvement in the health system includes the presence of a shura-e-sehie (village health council), its level of activity and whether minutes of their meetings are kept. This domain includes the following indicators: Overall Patient Satisfaction Patient Perception of Quality Index Written Shura-e-sehie activities in community Domain B: Staff Results This domain captures issues related to health facility staff. It includes staff satisfaction and perceptions of the work environment, as well as a measure of whether staff salary payments are current. This domain includes the following indicators: Health Worker Satisfaction Index Salary payments current Domain C: Capacity for Service Provision This domain refers to the capacity of a facility to provide high-quality health services. It includes input or structure-level indicators of a facility s readiness to provide services, including whether trained staff are in place, whether drug stocks are in place, what the condition of the physical infrastructure of the facility is, whether functional equipment and supplies are in place, whether the HMIS reporting and record keeping system is functional, and whether administrative processes are in place. This domain includes the following indicators: Equipment Functionality Index Drug Availability Index Page # 6

8 Family Planning Availability Index Laboratory Functionality Index (Hospitals & CHCs) Staffing Index -- Meeting minimum staff guidelines Provider Knowledge Score Staff received training in last year HMIS Use Index Clinical Guidelines Index Infrastructure Index Patient Record Index Facilities having TB register Domain D: Service provision Whereas the previous domain looks at readiness to provide services, this domain measures actual provision of services. This covers two main categories: clinical quality of care and health service outputs. This domain includes the following indicators: Patient History and Physical Exam Index Patient Counseling Index Proper sharps disposal Average new outpatient visits per month (BHC>750 visits) Time spent with patient (> 9 minutes) BPHS facilities providing antenatal care Delivery care according to BPHS Domain E: Financial Results This domain measures financial systems, including presence of user fee guidelines and exemption mechanisms for poor patients. This domain includes the following indicators: Facilities with user fee guidelines Facilities with exemptions for poor patients Domain F: Overall vision This domain reflects the overall vision and values of MOPH. This includes equity in service provision, gender barriers to care-seeking and protection of vulnerable groups in service provision. This domain includes the following indicators: Females as % of new outpatients Outpatient visit concentration index Patient satisfaction concentration index Page # 7

9 2.6 Collection of Data Instruments and Survey tools After the basic framework and indicators were developed, the JHU/IIHMR team developed, tested and finalized instruments for data collection (Annex I). The following instruments were developed for collecting data from health facilities and communities for the BSC: F1: Observations of Patient-provider interactions involving Patients under age five F2: Observations of Patient-provider interactions involving Patients five years of age and older F3: Exit interviews with the caretakers of patients under age five F4: Exit interviews with patients five years of age and older (or their caretakers) F5: Health Worker interviews (for facility-based health workers) F6: Community Health Worker interviews (only between ) F7: Facility assessment form C1: Household survey instrument (only in 2004) F8: Inpatient (IPD) interviews at District Hospitals (added in 2007) F9: Exit interviews with Inpatients (IPD) of District Hospitals (added in 2007) The first round of data was collected during the NHSPA conducted in Since that time, additional rounds of data collection have been conducted once a year. The survey is conducted during the summer of each year, involving staff from the Ministry s M&E Department and other technical staff from the Ministry and outside organizations. In total, over 350 people have been involved in data collection during each round. During the 2004 and 2005 rounds, data analysis was conducted by the JHU/IIHMR team, but from 2006 onwards MoPH staff started assuming an increasingly prominent role in analysis of data. A two-step approach was adopted in conducting training for data collectors. A team of master trainers was trained by facilitators from JHU/IIHMR on survey methods, administration of assessment instruments and other field protocols. These master trainers went to the regional level to train the data collectors in various regions around the country. For management of the assessment, Afghanistan was divided into six different regions comprising of four to seven provinces in each region. Collecting and recording the information Sampling Techniques A sampling frame of BPHS health facilities was created for each province by compiling a list of health facilities from MoPH in Kabul, and updating them with the Provincial Public Health Directors (PPHD), HMIS officers and key informants from NGOs and the MoPH in each province. Facilities deemed unsafe to visit were removed from the sampling frame. A stratified random sample of 25 facilities providing BPHS services was taken from each province. After finalizing the list of functional facilities, in consultation with the PPHDs, HMIS officers and NGOs, facilities were stratified into three groups District Hospitals (DH), Comprehensive Health Centers (CHC) and Basic Health Centers (BHC). For each province, facilities were then randomly Page # 8

10 selected from each stratum, using Microsoft Excel s RAND (random selection of a number in a specified range) function, according to the following distribution by facility type: 3 DHs 7 CHCs 15 BHCs Two randomly selected back-up sites were selected to replace facilities that could not be surveyed due to security or other reasons. If fewer than the above number of a particular facility type existed in any province, other facility types were substituted. If there were less than three DHs in any province, CHCs were substituted; if there were less than 7 CHCs, BHCs were substituted; and if BHCs were less, then 15 CHCs or DHs were substituted. If fewer than 25 active health facilities existed in any province, all facilities providing BPHS services were surveyed. During the survey, non-functional facilities were visited twice and if on the second visit there was no sign of the facility being operational, the randomly sampled replacement facility was surveyed instead, if of the same type. If the appropriate substitute facility type did not exist, a randomly selected facility of a different type was surveyed. The sampling scheme for patients was determined after the team arrived at a healthy facility. After the expected number of new outpatients for the day was estimated by the facility in-charge, a patient sampling scheme was determined, as described in Table 1. Ten patient-provider interactions were observed in each health facility: five with patients under age 5 and five with patients 5 years of age or older. Exit interviews were conducted with the same patients (or their caretakers) at the conclusion of their visit to the health facility. Table 1: Determining sampling pattern on basis of expected number of new outpatients Expected # of new patients in age stratum (under five years Sampling pattern vs. five years or older) If less than 10 patients, in each age stratum, are expected in a day If patients, in each age stratum, are expected in a day If patients, in each age stratum, are expected in a day If more than 20 patients, in each age stratum, are expected in a day Select each eligible patient until 5 observations of consultations involving patients in that age stratum have been completed Sample every second eligible patient until 5 observations of consultations involving patients in that age stratum have been completed Sample every third eligible patient until 5 observations of consultations involving patients in that age stratum have been completed Sample every fourth eligible patient until 5 observations of consultations involving patients in that age stratum have been completed The facility surveyors interviewed four facility-based health workers in each of the selected Basic Health Centers, Comprehensive Health Centers and Districts Hospitals. Field work The data for the BSC are taken from the National Health Services Performance Assessment (NHSPA), which is conducted each year between June and September. Data is collected on each of the instruments and collated to prepare the BSC. Page # 9

11 The final samples in 2008 NHSPA included assessments of over 600 health facilities, nearly 6,000 direct observations of patient-provider interactions and exit interviews, and interviews with over 2200 health workers. Table 2: Sample size for NHSPA Provinces Facilities Direct Observations of Patient Provider Interactions Patient Exit Interviews Health Worker Interviews CHW Interviews NA Supervision and Quality assurance Supervision of survey teams was done to ensure that data were collected in a complete and correct manner. Steps were taken at various stages for the same. Once the data collection had started the master trainers took the role of monitors and spot checks were conducted in all the provinces. Ten percent of facilities were re-sampled and the facility instrument was re-administered by an independent monitor. These results were compared to the data collected by the survey team, in order to assess the reliability of the data collected. The data quality assurance protocol followed involved several layers. Survey teams are trained in two steps. At first the M&E officers, from the MoPH, were trained centrally by the core team from JHU/IIHMR & MoPH. These trained M&E officers were sent as master trainers to different regions across Afghanistan to train survey teams from provinces. Monitoring Mechanism Various layers of monitoring mechanism were used to ensure the best quality of data collection from the sampled facilities. One extra member was added to the team of data collectors as a field editor. The specific duties of the editor are to monitor interviewer performance. Close supervision of interviewers and editing of completed interviews was essentially done to ensure accurate and complete data collection. Once a form was completed at the facility it was checked by the editor on the spot and any errors were corrected while the team was still at the facility. It was made mandatory that editing must be completed prior to leaving the facility. To the extent possible, the field supervisor would assist the editor in performing this task so that all interviews are field edited while still at the facility. After completion of the data collection process by the survey team postmonitoring was conducted in 35 per cent of the total sampled facilities in all provinces. For this activity special trained monitors were put on job. The monitors use Monitoring F7 formats for recording their observations without having access to the previously collected data by the survey team. Following is the list of activities done/followed as part of monitoring mechanism in the process of data collection: 1) The survey team visited the facility and collected the information as scheduled. 2) Once the survey was complete the Regional Manager established telephonic communication with the supervisor of the respective team. 3) The Regional Manager filled the Monitoring F7 KEY exactly as per the responses spoken by the supervisor over the telephone. Page # 10

12 4) Three variants of the Monitoring F7 Key are provided in this booklet. The Regional Manager would switch off occasionally between the versions so that teams do not have a chance to know which questions they were being monitored on. 5) The Regional Manager would then send a Monitor to the health facility to collect the information on the Monitoring F7 as part of Post- Monitoring exercise. 6) The Monitor would communicate and submit the entries of the Monitoring F7 to the Regional Manager to communicate. 7) The Regional Manager would compare the Monitoring F7 Key and Monitoring F7 and along with the Monitor would look for possible explanations of the discrepancies detected. The variants have more questions than what are asked in the Monitoring F7 forms by the Monitors. They need to ask ALL questions in the variants, but when comparing with the original Monitoring F7 forms, they only need to analyze the starred questions 8) In case significant discrepancies are found during the comparison, the Regional Manager would report the matter to the Central Office and seeks advice on the action to be taken on the matter. The Central Office would check the information using various resources for identifying the reasons of discrepancies and could send another special Monitor to the facility to verify the data and reasons of discrepancies. In worst cases a re-survey could be get done of the facilities where discrepancies are enormous. 2.7 Data Management Four main steps were followed in management of data collected from the field. These steps focused on management of survey forms and the data they contain, once the forms have been completed in the field, to final entry of the data. These steps have been elaborated as follows: Step I: Preparation of forms at the field level Step II: Collection and registration of the forms at MOPH office Step III: Editing of the forms Step IV: Data Entry Step I: Preparation of forms at the field level Before sending the forms to the Kabul Central Office, the Regional Team Managers ensured that all the forms were checked and verified by the team supervisor and editor. Moreover, the three F7 forms that were re-administered at three randomly sampled facilities in each province were compared to the F7 form filled out by the survey team, in order to assess the reliability of the data collected. All the Regional Team Managers (RTMs) ensured that the forms were verified by the field editor and the cover sheet properly completed and attached to the set of completed forms from each facility. All those cover sheets which did not have the required information were not sent to Kabul office till the requisite information was there. Step II: Collection and registration of the forms at MOPH office Submission of complete and field edited forms at the Monitoring and Evaluation Department at the MOPH was the primary responsibility of the Regional Team Managers. An Assistant Data Officer was appointed for receiving and archiving the forms in the record room. He was responsible for safekeeping of the forms. Tracking sheets were developed (attached with editing guidelines) for receiving the data and a new cover sheet for each of the facilities was prepared. Movement of forms for editing and entry was recorded in a separate tracking sheet. Page # 11

13 As far as possible, once submitted, the forms were not allowed to be taken outside the MOPH office building. All the forms used for data management were maintained both in hard copy and soft copy. The Assistant Data Management Officer was responsible for updating the database for these forms. Step III: Editing of the forms Date editing, at the office (referred to as office editing) ensures those checks that must be carried out to verify the internal consistency of responses to questions in the questionnaires. Before being processed for data entry, all the forms were office edited by a team of trained data editors. The office editing guidelines are attached as Annex II. A Data Manager was responsible for overall management of the data editing. He ensured that the data editing team followed the data editing guidelines. The forms which were removed from the record room were duly registered using the format prescribed and this was supervised by the Data Manager. The data tracking sheets are attached as part of the data editing guidelines. Office editing also ensures that each form is assigned a unique ID. The process of assigning unique IDs to each form is detailed in the editing guidelines. Each editor is required to maintain a data editing sheet, which is handed over to the Assistant Data Manager at the end of each work day. The data manager randomly re-checked five percent of the survey forms, in order to assess the accuracy and completeness of the data editing process. Any form that had more than two errors led to the rejection of the whole batch and that batch was re-edited by another team. After editing was completed, the Assistant Data manager handed the forms over to the data entry team. Step IV: Data Entry It is essential that the data entry process is of high quality, since an error here can negate the hard work done by all others. All results are based on the data entered so these steps assume enormous importance. The software used for data entry is Census and Survey Processing System (CSPro). It is a software package for entering, editing, tabulating, and disseminating data from censuses and surveys. It is a public domain product and can be used, distributed and downloaded from at no cost. A three-step data entry method was followed for the NHSPA. All the forms were entered by a team of trained entry operators. These forms were double entered and verified by a different set of trained entry operators to minimize error during the first round of data entry. Once the two datasets were ready, they were checked for consistency. Any inconsistencies between the two datasets were resolved by looking at the original forms and one final dataset was created on this basis. Each data entry operator maintained a data entry sheet and this sheet was given to the Assistant Data Manager for updating every evening. The Data Manager was personally responsible for ensuring the quality of the information. Before finishing data entry for a province, he re-checked 5 per cent of the data himself. This included checking at least one F7, seven F1/F3, seven F2/F4 and four each of F5 and F6. Any form which was found to have more than two errors led to rejection of the whole batch and that batch was reentered by another team. The Computer Programmer was responsible for taking a daily back up of the data entered and copying the same onto his computer and the Data Manager s computer at the end of each work Page # 12

14 day. For taking a back-up, it is advisable to make separate folders for every day with sub-folders for every entry operator wherein he should save the data for each province in a separate folder. It is the responsibility of the computer programmer to hand over the complete data set for each of the forms after merging all the facilities and provinces. 2.7 Data Analysis For the first two rounds, data analysis was conducted by the JHU/IIHMR team, with feedback from the Monitoring and Evaluation Advisory Board. From 2006 onwards MOPH staffs assumed prominent role in analysis of data. The statistical software package Stata 9 was used to conduct the analysis (Stata Corp: Stata Statistical Software: Release 9.0, Stata Corporation, College Station, TX, April, 2005). The provincial results for each indicator were weighted according to a standardized distribution of important characteristics. Most indicators were weighted according to a standardized distribution of facility type, while a few indicators were weighted according to a standardized distribution of health worker type and one indicator was weighted according to a standardized distribution of health worker gender (see table 2 & 3). The M&E Advisory Board, along with the JHU/IIHMR team, conducted a sensitivity analysis to determine whether the overall results are sensitive to the weighting given to each indicator on the scorecard. The MOPH, the M&E Advisory Board and the JHU/IIHMR team concluded that all indicators on the scorecard would be given equal weighting for two reasons: 1) the composite score assigned to each province was found to have a low sensitivity to changes in the weighting schemes applied to the indicators on the scorecard (that is to say, the composite scores did not change very much when different weighting schemes were applied); and 2) developing a differential weighting scheme involves a great deal of subjectivity and different individuals were found to have different opinions regarding which indicators should be given what weights. 2.8 Disseminating the scorecard After the M&E Advisory Board finalized the BSC, the scorecard was presented to Ministry of Public Health, Afghanistan (MoPH) - Technical Advisory Group, which formed a committee to advise MOPH leadership on actions to be taken on the basis of evidence presented in the BSC. Thereafter a series of presentations on the BSC and its findings from successive rounds of NHSPA data collections were made in English, Dari and Pashto from 2004 to 2008 at various platforms, including the following: National Technical Co-ordination Committee (NTCC) Consultative Group on Health and Nutrition PPA/EC Managers Meeting Provincial Public Health Directors Quarterly Meeting PPA/EC Quarterly Meetings HMIS officers and provincial advisors of REACH Project In addition, a series of discussions were held with MOPH leadership and technical staff, provinciallevel staff, donors and implementers to provide further elaboration on how the scorecard and the findings it generates can be used to improve management of BPHS implementation. The BSC has proved to be useful for managers and policymakers working in Afghanistan s health sector. The BSC has helped MOPH leadership, technical staff, donor agencies, BPHS Page # 13

15 implementers and other partners to maintain a focus on achieving measurable results. It has also provided an evidence base on areas of strength and weaknesses and facilitated the benchmarking of performance in service delivery. Page # 14

16 SECTION 3: DETAILED DESCRIPTION OF DOMAINS AND INDICATORS This section defines the domains and indicators used in the BSC, including details of how they were measured and analyzed. Most results were weighted according to either the distribution of BPHS health facilities or the distribution of health workers in the national sample. Weighting allows more meaningful comparisons across provinces. In cases where indices were created out of multiple items, missing values were dropped from the analysis, and only complete scores were used. All cut-offs/benchmarks used in all BSC (2004 onwards) were calculated from the 2004 national sample, facilitating temporal comparisons. Setting Benchmarks Benchmarking has been introduced in the BSC as a performance improvement tool whereby NGOs or Provincial Public Health Departments (PPHDs) engaged in delivery of services in different provinces can measures its performance against other provinces and determine how these provinces achieved their performance levels. They can also study best practices models and use the information to improve its own performance. At the national level upper and lower benchmarks have been set for each indicator based on available data from each province. In 2004, after scores were calculated for each province they were arranged in descending order, for every indicator, and upper and lower levels were calculated as mentioned below: Upper Benchmark The upper benchmarks has been set at the minimum level of performance achieved by the provinces in the top quintile in 2004 i.e. if we arrange the provinces in descending order of scores the 6th province from the top out of a total of 33 in 2004 is our upper benchmark. Lower Benchmark The lower benchmarks has been set at the minimum level of performance achieved by the provinces in the bottom quintile in 2004 i.e. if we arrange the provinces in descending order of scores the 27th province from the top out of a total of 33 in 2004 is our lower benchmark. GENERAL INSTRUCTIONS FOR READING EXAMPLES/DEMONSTRATIONS FOR CALCULTING THE SCORES 1. For most of the calculations the original variable has been renamed before recoding or conducting analysis to retain the original set of variables and properties of individual variable. 2. Information given in parenthesis in the tables is STATA output generated by the changes made to the variable by the command. 3. The numbers mentioned in the demonstrations are based on the NHSPA 2008 and may vary if similar commands are used on a different dataset. 4. Sample calculations are shown for all the indicators and the final number that would be recorded in the BSC is in bold. The purpose of these sample calculations is for illustration only. 5. Indicators were calculated in statistical programs such as STATA with weighting done in Microsoft Excel. Results were then pasted into the colored template illustrated in Table 1 & 2. Page # 15

17 STEPS FOR WEIGHTING OF SCORES IN A PROVINCE Mean scores were calculated for each health facility type present, for each province. These scores were weighted by multiplying it by the standardized distribution of important characteristics. Details of weights used for each indicator has been given in the table below: Table 2: Description of types of weights used for each indicator in the Balanced Scorecard # Domains and Indicators Type of Weight A. Patients & Community 1 Overall Patient Satisfaction Health Facility Type 2 Patient Perception of Quality Index Health Facility Type 3 Written Shura-e-sehie activities in community Health Facility Type B. Staff 4 Health Worker Satisfaction Index Health Worker Type 5 Salary payments current Gender of Health Worker C. Capacity for Service Provision 6 Equipment Functionality Index* Health Facility Type 7 Drug Availability Index Health Facility Type 8 Family Planning Availability Index Health Facility Type 9 Laboratory Functionality Index (Hospitals & CHCs) Health Facility Type 10 Staffing Index -- Meeting minimum staff guidelines Health Facility Type 11 Provider Knowledge Score Health Worker Type 12 Staff received training in last year Health Worker Type 13 HMIS Use Index Health Facility Type 14 Clinical Guidelines Index Health Facility Type 15 Infrastructure Index Health Facility Type 16 Patient Record Index Health Facility Type 17 Facilities having TB register Health Facility Type 18 Patient History and Physical Exam Index Health Facility Type 19 Patient Counseling Index Health Facility Type 20 Proper sharps disposal Health Facility Type 21 Average new outpatient visit per month (BHC > 750 visits) Health Facility Type 22 Time spent with patient (> 9 minutes) Health Facility Type 23 BPHS facilities providing antenatal care Health Facility Type 24 Delivery care according to BPHS Health Facility Type E. Financial Systems 25 Facilities with user fee guidelines Health Facility Type 26 Facilities with exemptions for poor patients Health Facility Type F. Overall Vision 27 Females as % of new outpatients Health Facility Type 28 Outpatient visit concentration index Health Facility Type 29 Patient satisfaction concentration index Health Facility Type Page # 16

18 Table 3: Calculation of standardized distribution of weights used in the NHSPA 2008 Health Facility Type Health Worker Type Table 1.1a Table 1.2a Gender of Health Worker All 3 types present N % Weight All 5 category present n % Weight Table 1.3a BHC Doctors Health Worker n % Weight CHC Nurse Male DH Midwife/ Aux. Midwife Female Total Assn. Doctor Total Others Table 1.1b Total BHC missing N % Weight CHC Table 1.2b DH Others missing n % Weight Total Doctors Nurse Midwife/ Aux. Midwife Table 1.1c Assn. Doctor CHC missing N % Weight Total BHC DH Table 1.2c Total Assn. Doctor missing n % Weight Doctors Nurse Table 1.1d Midwife/ Aux. Midwife DH missing N % Weight Others BHC Total CHC Table 1.2d Total Assn. Doctor & Others missing n % Weight Doctors Nurse Midwife/ Aux. Midwife Total Table 1.2e Nurse missing n % Weight Doctor MW/Aux MW Assistant Dr Others Total Page # 17

19 Instructions For BHCs = (Average % score in BHC in a province for indicator 1 ) X (weight for BHC) For CHCs = (Average % score in CHC a province for indicator 1 ) X (weight for CHC) For DHs = (Average % score in DH a province for indicator 1 ) X (weight for DH) If any one type of facility is missing in a province then multiply the scores with respective weights for the missing type of facility. If all three types of health facilities are present in a province, use weights in table 1.1a for weighing facility scores. If BHCs are missing in a province, use the weights in table 1b for weighing respective facilities scores. If CHCs are missing in a province, use the weights in table 1c for weighing respective facilities scores. If DHs are missing in a province, use the weights in table 1d for weighing respective facilities scores. After weighting the individual scores for each facility type in a province add the scores to get the score for the province. =(weighted value for BHC + weighted value for CHC + weighted value for DH) x 100 Due to the uncertainty regarding the number of active BPHS facilities in 2004, the sampling fraction (that is, the relative proportion of each facility type among all facilities in the sample) was used for adjustment of scores instead of the population fraction (that is, the relative proportion of each facility type among all BPHS facilities in the country). In 2005 and 2006, the sampling fraction was used again in order to maintain comparability with A sensitivity analysis was conducted to see whether the scores would change much if the population fraction were used for 2005 and 2006 instead of the sampling fraction. The results of the analysis showed almost no change in provincial or national scores when the population fraction was used instead of the sampling fraction. When the BSC indicators are revised to reflect the evolving information needs of the health sector, a switch will be made to the population fraction. 3.1 Domain A: Patients and Community Description This domain captures important issues related to clients of health services and other community members. In each of the health facility visited for assessment, caretakers of five under-five patients and five over-five patients attending the health facility were interviewed using an exit interview form and asked about their visit. It includes patient satisfaction, community perceptions of the level of quality of available services, levels of community involvement in the health system and levels of community awareness of service availability. Indicators of community involvement in the health Page # 18

20 system include the presence of a shura-e-sehie, its level of activity and whether minutes of their meetings are kept. Aims To improve community and patient perceptions of the availability and quality of health services To ensure that communities are involved in important decisions on provision of health services Indicators 1. Overall Patient Satisfaction 2. Patient perceptions of Quality Index 3. Written record shura-e-sehie activities in community 1. Patient Satisfaction Description This indicator assesses patients overall level of satisfaction with their visit to the health facility. Patients were asked their level of agreement or disagreement with the statement, Your overall visit was satisfactory. The levels of agreement or disagreement were measured on a four point Likert s scale. To make it simple and understandable the respondents were asked this question rating their satisfaction by assigning 1-4 naans. The four possible responses to this question were: very unsatisfied (1), unsatisfied (2), satisfied (3) and very satisfied (4). Technical Details Question #152 from F1/F3 and F2/F4 were used to calculate this indicator. A percentage score was calculated for each exit interview conducted. The four possible scores for each exit interview were 0%, 33.33%, 66.66% and 100% corresponding to four possible responses, which were, 1, 2, 3 and 4 respectively. This was done by reducing each individual score by 1 point and dividing it by 3. Average percentage score was calculated for each facility by averaging the score over all exit interviews conducted in a facility. These scores were then averaged over each facility type in a province by getting an average score for all BHC s, CHC s and DH s in a province. These average score for each facility type in a province were then weighted to make these score nationally comparable. These weights were the proportion of each facility type in the national sample. Final score for each province was calculated by adding the percentage scores across facility type for each province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 1 USING STATA TO PRODUCE SCORES FOR INDICATOR 1: OVERALL PATIENT SATISFACTION Page # 19

21 STEP 1: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEM FROM F1/F3 DATA SET gen satisfac=f3q152 (20 missing values generated) STEP 2: RECODE NO RESPONSES/ DON T KNOW AS MISSING recode satisfac (98/99=.) (satisfac: 5 changes made) STEP 3: REDUCE THE VALUES FOR PATIENT SATIFACTION BY 1 AND DIVIDING BY 3 TO CONVERT IT TO A PERCENTAGE gen sat = satisfac-1 (25 missing values generated) gen i1 = sat/3 (25 missing values generated) STEP 4: LABELING OF VARIABLES label var sat "My overall visit was satisfactory" STEP 5: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEM FROM F2/F4 DATA SET gen satisfac=f4q152 STEP 6: RECODE NO RESPONSES/ DON T KNOW AS MISSING recode satisfac (98/99=.) (satisfac: 4 changes made) STEP 7: REDUCE THE VALUES FOR PATIENT SATIFACTION BY 1 AND DIVIDING BY 3 TO CONVERT IT TO A PERCENTAGE gen sat = satisfac-1 (4 missing values generated) gen i1 = sat/3 (4 missing values generated) STEP 8: LABELING OF VARIABLES label var sat "My overall visit was satisfactory" STEP 9: APPEND VAR i1 GENERATED FROM F1/F3 AND F2/F4 use "f1f3 vars.dta", clear append using "f2f4 vars.dta" Page # 20

22 STEP 10: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean satisfac) row Un-weighted Scores Province BHC CHC DH Kabul MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN IN TABLE 1.1a Weighted Scores Province BHC CHC DH Kabul ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET MEAN PROVINCIAL SCORES Province score Province Score Kabul Patient Perceptions of Quality Index Description This indicator measures patient perceptions of quality of care. The indicator includes nine items. Similar to indicator one agreement or disagreement for each of the nine items was measured on the naan scale rating from 1 to 4 naan, with 1 representing complete disagreement and 4 indicating complete agreement. The list of the nine items has been given below: 1. Convenience of travel to health facility 2. Cleanliness 3. Courtesy and respect of staff 4. Trust in skills and abilities of health workers 5. Explanation of illness 6. Explanation of treatment 7. Ease of getting prescribed medications 8. Reasonableness of cost 9. Adequacy of privacy Technical Details This indicator is calculated from questions # from F3&F4 in Scores on each of these 9 items were combined such that every exit interview had a score ranging from Page # 21

23 Score for each exit interview was reduced by 9 points such that an exit interview with one (very unsatisfied) for all nine questions could be considered as 0 and an exit with four (very satisfied) on every question could be considered as 27. The effective range was reduced to A percentage score was calculated for each exit interview conducted by dividing each individual score by 27. Average percentage score was calculated for each facility by averaging the score over all exit interviews conducted in a facility. These scores were then averaged over each facility type in a province by getting an average score for all BHC s, CHC s and DH s in a province. These average score for each facility type in a province were then weighted to make these score nationally comparable. These weights were the proportion of each facility type in the national sample. Final score for each province was calculated by adding the percentage scores across facility type for each province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 2 USING STATA TO PRODUCE SCORES FOR INDICATOR 2: PATIENT PERCEPTION OF QUALITY INDEX STEP 1: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS FROM DATA SET F1/F3 gen sat_travel=f3q143 (20 missing values generated) gen sat_clean=f3q144 (20 missing values generated) gen sat_courtesy=f3q145 (20 missing values generated) gen sat_skills=f3q146 (20 missing values generated) gen sat_explainill=f3q147 (20 missing values generated) gen sat_explainrx=f3q148 (20 missing values generated) gen sat_meds=f3q149 (20 missing values generated) gen sat_cost=f3q150 (20 missing values generated) gen sat_privacy=f3q151 (20 missing values generated) STEP 2: RECODING NO RESPONSES/ DON T KNOW AS MISSING recode sat_travel (98/99=.) (sat_travel: 2 changes made) recode sat_clean (98/99=.) (sat_clean: 2 changes made) recode sat_courtesy (98/99=.) Page # 22

24 (sat_courtesy: 6 changes made) recode sat_skills (98/99=.) (sat_skills: 5 changes made) recode sat_explainill (98/99=.) (sat_explainill: 4 changes made) recode sat_explainrx (98/99=.) (sat_explainrx: 2 changes made) recode sat_meds (98/99=.) (sat_meds: 6 changes made) recode sat_cost (98/99=.) (sat_cost: 5 changes made) recode sat_privacy (98/99=.) (sat_privacy: 2 changes made) STEP 3: GENERATING NEW VARIABLE FOR PATIENT PERCEPTION OF QUALITY INDEX BY ADDING NINE INDIVIDUAL SATISFACTION VARIABLES gen ptperq = sat_travel + sat_clean + sat_courtesy + sat_skills + sat_explainill + sat_explainrx + sat_meds + sat_cost + sat_privacy (37 missing values generated) STEP 4: REDUCE THE VALUES BY 9 AND DIVIDE BY 27 TO CONVERT IT TO A PERCENTAGE gen ptper = (ptperq-9)/27 (37 missing values generated) STEP 5: LABELING OF VARIABLES label var sat_travel "It is convenient to get from my house to the health unit" label var sat_clean "The health unit is clean" label var sat_courtesy "The health staff are courteous and respectful" label var sat_skills "I have trust in the skills and abilities of the health workers" label var sat_explainill "The health workers did a good job of explaining the illness" label var sat_explainrx "The health workers did a good job of explaining the treatment" label var sat_meds "It is easy to get medicines that health workers prescribe" label var sat_cost "The cost of this visit to the health unit was reasonable" label var sat_privacy "I had enough privacy during my visit" label var ptperq "Patient Perceptions of Quality Index" STEP 6: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS FROM DATA SET F2/F4 gen sat_travel=f4q143 gen sat_clean=f4q144 gen sat_courtesy=f4q145 gen sat_skills=f4q146 gen sat_explainill=f4q147 gen sat_explainrx=f4q148 gen sat_meds=f4q149 gen sat_cost=f4q150 gen sat_privacy=f4q151 Page # 23

25 STEP 7: RECODING NO RESPONSES/ DON T KNOW AS MISSING recode sat_travel (98/99=.) (sat_travel: 0 changes made) recode sat_clean (98/99=.) (sat_clean: 2 changes made) recode sat_courtesy (98/99=.) (sat_courtesy: 6 changes made) recode sat_skills (98/99=.) (sat_skills: 0 changes made) recode sat_explainill (98/99=.) (sat_explainill: 1 changes made) recode sat_explainrx (98/99=.) (sat_explainrx: 2 changes made) recode sat_meds (98/99=.) (sat_meds: 1 changes made) recode sat_cost (98/99=.) (sat_cost: 3 changes made) recode sat_privacy (98/99=.) (sat_privacy: 0 changes made) STEP 8: GENERATING NEW VARIABLE FOR PATIENT PERCEPTION OF QUALITY INDEX BY ADDING NINE INDIVIDUAL SATISFACTION VARIABLES gen ptperq = sat_travel + sat_clean + sat_courtesy + sat_skills + sat_explainill + sat_explainrx + sat_meds + sat_cost + sat_privacy (14 missing values generated) STEP 9: REDUCE THE VALUES BY 9 AND DIVIDE BY 27 TO CONVERT IT TO A PERCENTAGE gen ptper = (ptperq-9)/27 (14 missing values generated) STEP 10: APPEND VAR i2 GENERATED FROM F1/F3 AND F2/F4 use "f1f3 vars.dta", clear append using "f2f4 vars.dta" STEP 11: TABULATING OVERALL PERCEPTION VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean ptperq) row Un-weighted Scores Province BHC CHC DH Kapisa MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN ON TABLE 1.1a Weighted Scores Province BHC CHC DH Page # 24

26 Kapisa ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province score Province Score Kapisa Written shura-e-sehie activities in the community Description This indicator measures the level of community involvement in management of the health facility. Shura-e-sehie are village health committees established by health providers in their respective facilities and have involvement of members from the community and facility staff. This index measures the percentage of facilities with a written record of shura-e-sehie activities conducted in the community. The importance of this is two-fold. It is assumed that the existence of a written document at the health facility is associated with an active shura-e-sehie, linked to the health facility. Technical Details A percentage was calculated of those responding yes to Question #1301 in F7. This percentage was weighted according to the national sample of health facility types. STATA DEMONSTRATION 3 USING STATA TO PRODUCE SCORES FOR INDICATOR 3: WRITTEN SHURA-E-SEHIE ACTIVITIES IN THE COMMUNITY STEP 1: GENERATING DUPLICATE VARIABLE gen i3=1 if q1301==1 & q1304==1 (69 missing values generated) replace i3=0 if q1301==2 & q1304==2 (0 real changes made) replace i3=0 if q1301==1 & q1304==2 (30 real changes made) replace i3=0 if q1301==2 & q1304==. (34 real changes made) replace i3=0 if q1301==2 & q1304==1 (0 real changes made) STEP 2: RECODE NO RESPONSES AS MISSING Page # 25

27 recode q1304 (98=.) (q1304: 3 changes made) STEP 3: TABULATING WRITTEN SHURA-E-SEHIE RECORD VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean i3) Un-weighted Scores Province BHC CHC DH Kapisa MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN IN TABLE 1.1a Weighted Scores Province BHC CHC DH Kapisa ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET MEAN PROVINCIAL SCORES Province score Province Score Kapisa Domain B: Staff Description This domain captures issues related to health facility staff. In each of the health facility a maximum of four different types of health worker were interviewed. It includes staff satisfaction and current levels of salary payment. The number of different types of health workers working in facilities (doctors/nurses/midwives/vaccinators, availability of female staff) are not included in this category. Aims Improve staff satisfaction with their working environment Ensure provision of appropriate staff incentives, including salaries, working conditions, required training, and relationships with colleagues Indicators 4. Health Worker Satisfaction Index 5. Salary payments current Page # 26

28 4. Health worker Satisfaction Index Description The indicator measures the level of satisfaction of the health worker working in each facility. This indicator measures the satisfaction of the health worker under the following 19 items. Similar to indicator one and two agreement or disagreement for each of the nine items was measured on the naan scale rating from 1 to 4 naan, with 1 representing complete disagreement and 4 indicating complete agreement. 1. Working relationship with facility staff 2. Working relationship with provincial MoPH staff 3. How well the MoPH or NGO facility is managed 4. Relationship with local leadership 5. Availability of medicines 6. Availability of equipment 7. Physical condition of health facility 8. Availability to provide high quality care 9. Respect and standing in the community 10. Training opportunities to upgrade skills 11. Ability to meet needs of the community 12. Salary 13. Employment benefits 14. Security 15. Living accommodation for your children 16. Access to education for your children 17. A boss who recognizes your good work 18. Opportunities for promotion 19. Overall job satisfaction Technical Details Questions # in F5 were used for this indicator. For this index, the score for each province was weighted according to a standardized national distribution of health worker type rather than type of health facility. A simple index was created by combining the 19 items. Missing and no response on any of the 19 items lead to dropping of that health worker from the analysis for this index. Five /types categories of health workers were considered for this index, namely doctor, nurse, midwife/auxiliary midwife, assistant doctor and others. For each health worker category, average score on each of the 19 items was calculated. These 19 scores were added together for each health worker category to give a score ranging from Score for each health worker was reduced by 19 points such that a health worker with one (very unsatisfied) for all nine questions could be considered as 0 and a health worker with four (very satisfied) on every question could be considered as 57. The effective range was reduced to A percentage score was calculated for each health worker conducted by dividing each individual score by 57. Average percentage score was calculated for each facility by calculating average score for each of the five categories of health workers interviewed in a facility. These scores were then averaged over each health worker type in a province by getting an average score for every health worker category in a province. Page # 27

29 These average score for each health worker in a province were then weighted to make these score nationally comparable. These weights were the proportion of every category of health workers in the national sample. Final score for each province was calculated by adding the percentage scores across health worker type for each province. STATA DEMONSTRATION 4 USING STATA TO PRODUCE SCORES FOR INDICATOR 4: HEALTH WORKER SATISFACTION STEP 1: GENERATING DUPLICATE VARIABLE FOR EACH WORKER POSITION gen post =. (2233 missing values generated) FOR DOCTOR replace post = 1 if q107==1 (548 real changes made) FOR NURSE replace post=2 if q107==2 (380 real changes made) FOR MIDWIFE/ AUX. MIDWIFE replace post = 3 if (q107 == 3 q107==4) (493 real changes made) FOR ASSISTANT DOCTOR replace post = 4 if q107==5 (27 real changes made) FOR OTHERS replace post = 5 if q107==6 q107==7 (756 real changes made) STEP 2: LEVEL AND DEFINE VARIABLE label var post "Health worker position for analysis" label define pq103 1 "Doctor" 2 "Nurse" 3 "Midwife/Aux.Midwife" 4 "Assistant Doctor" 5 "Other" label value post pq103 STEP 3: RECODING OF VARIABLES INCLUDING NO RESPONSES/MISSING recode q174 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q174: 2233 changes made) recode q175 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q175: 2233 changes made) recode q176 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q176: 2233 changes made) recode q177 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q177: 2233 changes made) Page # 28

30 recode q178 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q178: 2233 changes made) recode q179 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q179: 2233 changes made) recode q180 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q180: 2233 changes made) recode q181 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q181: 2233 changes made) recode q182 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q182: 2233 changes made) recode q183 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q183: 2233 changes made) recode q184 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q184: 2233 changes made) recode q186 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q186: 2233 changes made) recode q187 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q187: 2233 changes made) recode q188 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q188: 2233 changes made) recode q189 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q189: 2233 changes made) recode q190 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q190: 2233 changes made) recode q191 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q191: 2233 changes made) recode q192 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q192: 2233 changes made) recode q193 (1=0) (2=1) (3=2) (4=3) (98/99=.) (q193: 2233 changes made) STEP 4: GENERATING NEW VARIABLE FOR HEALTH WORKER SATISFACTION gen hwsat = (q174 + q175 + q176 + q177 + q178 + q179 + q180 + q181 + q182 + q183 + q184 + q186 + q187 + q188 + q189 + q190 + q191 + q192 + q193) (470 missing values generated) STEP 5: DEVIDING THE VALUES FOR HEALTH WORKER SATIFACTION BY 57 TO Page # 29

31 GENERATE i4 gen i4 = (hwsat/57) (470 missing values generated) STEP 6: LABELING OF VARIABLES label var hwsat "Health worker Satisfaction Index" STEP 7: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND FACILITY TYPE table province post, c(mean i4) Un-weighted Scores Province Doctor Nurse Midwife/ Aux. Assn. Doc. Others Wardak MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN IN TABLE 1.2c Weighted Scores Province Doctor Nurse Midwife/ Aux. Assn. Doc. Others Wardak ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province Score Province Score Wardak Salary Payment Current Description This indicator illustrates the proportion of health workers who have received their salary within the month before the survey. Gender rather than health worker type was hypothesized to affect this indicator. Technical Details The percentage of healthcare workers answering yes to question # 116 from F5 was calculated and weighted by the distribution of health worker gender in the national sample. Note that CHWs were excluded in this indicator because they are not officially paid. Technical Details Questions #116 in F5 were used for this indicator. Page # 30

32 A percentage of healthcare workers answering yes to the question was calculated. For this index, the score for each province was weighted according to the distribution of health worker gender in the national sample. Average scores for were calculated by Male and Female distribution of health workers in the sample. These average score for each health worker in a province were then weighted to make these score nationally comparable. These weights the distribution of health worker gender in the national sample. Final score for each province was calculated by adding the percentage scores across health worker type for each province. STATA DEMONSTRATION 5 USING STATA TO PRODUCE SCORES FOR INDICATOR 5: SALARY PAYMENT CURRENT STEP 1: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS gen i5 = q116 STEP 2: RECODING NO RESPONSES/DONOT KNOW AS MISSING AND NOT AVAILABLE AS ZERO 0 recode i5 (98/99=.) (2=0) (i5: 526 changes made) STEP 3: LABELING OF VARIABLES label var i5 "salary up to date (Yes/ No) STEP 4: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND GENDER table province q105sex, c(mean i5) Un-weighted Scores Female Province Male HW HW Logar MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN IN TABLE 1.3a Weighted Scores Female Province Male HW HW Logar ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province Score Province Score Logar Domain C: Capacity for Service Provision Page # 31

33 Description This domain refers to the capacity of a facility to provide high quality of basic health services. It looks at various factors that contribute to the health facility s readiness to provide services. These factors include the presence of trained staff, clinical knowledge of the staff, adequate drug stocks, appropriate physical infrastructure, existence of equipment and supplies, adequate health management information system reporting, appropriate record keeping system and key administrative processes. Aims To ensure adequate numbers and types of staff who have appropriate skill levels To ensure adequate physical infrastructure, including buildings, drugs, equipment and communication systems Provide regular and timely support from management, such as supervision and on-the-job training Indicators 6. Equipment Functionality Index 7. Drug Availability Index 8. Family Planning Availability Index 9. Laboratory Functionality Index (Hospitals & CHCs) 10. Staffing Index Meeting minimum staffing guidelines 11. Provider Knowledge Score 12. Staff received training in last year 13. Health Management Information Systems (HMIS) Use Index 14. Clinical Guidelines Index 15. Infrastructure Index 16. Patient Record Index 17. Facilities having tuberculosis register 6. Equipment functionality index Description This indicator measures the presence of a minimal number of functional equipment required by a health facility so it can adequately provide the BPHS. A list of 14 essential equipments was decided, that should be present all the three levels of the health facility types. The index contains the following 14 items: 1. Children s scale 2. Height measure 3. Adult scale 4. Blood pressure cuff 5. Thermometer 6. Stethoscope 7. Otoscope 8. Sterilizer Page # 32

34 9. Section/aspiration device 10. Vision chart 11. Minor surgical set 12. Fetoscope 13. Speculum 14. Vaccine refrigerator Technical Details Questions # 500, , 518, 707, 815, 832 & 867 from F7 were used to calculate this indicator. For each item mentioned above that is present and is functional, a health facility received 1 point. If the item is not functional, irrespective of whether it is present or absent, the facility received 0 points. A simple index was calculated by adding scores on each of the 14 items for the facility. The range of scores possible for each facility is A percentage score was calculated by dividing the scores for each facility by 14. Average scores were calculated for each facility type in a province and were weighted by the distribution of health facilities in the national sample. These weighted scores were then added for each province to obtain the mean for that province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 6 USING STATA TO PRODUCE SCORES FOR INDICATOR 6: EQUIPMENT INDEX STEP 1: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS gen ncscale = q502 gen nhtmeas = q503 gen nascale = q504 gen nbpcuff= q505 gen ntherm= q506 gen nsteth = q507 gen noto= q508 gen nsterilizer= q500 gen nsuction= q509 gen nvischart= q510 gen nminsurgset= q518 gen nfetoscope1= q815 (36 missing values generated) gen nfetoscope2= q832 gen nspeculum= q867 gen nvacfridge= q707 (13 missing values generated) STEP 2: RECODING NO RESPONSES AS MISSING Page # 33

35 FOR ANY ITEM NOT PRESENT OR PRESENT AND NOT FUNCTIONAL, THE FACILITY GETS A SCORE OF 0 recode ncscale nhtmeas nascale nbpcuff ntherm nsteth noto nsterilizer nsuction nvischart nminsurgset nfetoscope1 nfetoscope2 nspeculum nvacfridge (98/99=.) (2/3=0) (ncscale: 68 changes made) (nhtmeas: 95 changes made) (nascale: 52 changes made) (nbpcuff: 10 changes made) (ntherm: 34 changes made) (nsteth: 4 changes made) (noto: 106 changes made) (nsterilizer: 109 changes made) (nsuction: 262 changes made) (nvischart: 163 changes made) (nminsurgset: 86 changes made) (nfetoscope1: 28 changes made) (nfetoscope2: 77 changes made) (nspeculum: 73 changes made) (nvacfridge: 12 changes made) STEP 3: VACFRIDGE RECODED TO ZERO IF FACILITY DOES NOT PROVIDE EPI OR DOES NOT HAVE A VACCINE FRIDGE replace nvacfridge=0 if q700==2 q711==1 (7 real changes made) STEP 4: CREATION OF FETOSCOPE VARIABLE IF WORKING FETOSCOPE IS PRESENT IN EITHER ANC CLINIC OR DELIVERY ROOM, FACILITY IS GIVEN CREDIT FOR HAVING FETOSCOPE gen fetoscope=0 replace fetoscope=1 if nfetoscope1==1 nfetoscope2==1 (580 real changes made) STEP 5: ADJUST FOR SPECULUM AND DELIV KIT MISSING replace nspeculum=0 if q819==2 (43 real changes made) STEP 6: GENERATING NEW VARIABLE FOR EQUIPMENT INDEX gen equipindex = ncscale + nhtmeas + nascale + nbpcuff + ntherm + nsteth + noto + nsterilizer + nsuction + nvischart + nminsurgset + fetoscope + nspeculum + nvacfridge (26 missing values generated) STEP 7: LABELING OF VARIABLES label var equipindex "Equipment index (0-14)" Page # 34

36 STEP 8: GENERATE i6 BY DIVIDING THE SCORES BY 14 TO CONVERT IT TO A PERCENTAGE gen i6 = equipindex/14 (26 missing values generated) STEP 9: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean i6) row center Un-weighted Scores Province BHC CHC DH Ghazni MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN ON TABLE 1.1a Weighted Scores Province BHC CHC DH Ghazni ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province Score Province Score Ghazni Drug availability index Description This indicator measures the availability of essential drugs at the facility level. A total of 5 drugs are included in this indicator. The surveyor asks for any stock-outs in the last 30 days for the following drugs: 1. Tetracycline ophthalmic ointment 2. Paracetamol tabs 3. Amoxicillin 4. ORS packets 5. Iron tablets Technical Details Questions # from F7 were used for this indicator. For each of the listed items that were in stock throughout the month prior to the survey, a facility received 1 point. If there was a stock-out in the month prior to the survey the facility receives a score of 0. A simple index is created by adding all the score for each facility. The range of scores possible for each facility is 0-5. A percentage score was calculated out of a maximum of 5 for each facility type within a province and weighted by the distribution of health facilities in the national sample. Page # 35

37 These weighted scores were then added for each province to obtain the mean for that province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 7 USING STATA TO PRODUCE SCORES FOR INDICATOR 7: DRUG AVAILABILITY INDEX STEP 1: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS gen tcyc = q602a gen paracet= q603a gen amox= q604a gen ors= q605a gen iron= q606a STEP 2: RECODING NOT AVAILABLE AS ZERO 0 AND NO RESPONSES/DONOT KNOW AS MISSING recode tcyc paracet amox ors iron (2=0) (98/99=.) STEP 3: GENERATING NEW VARIABLE FOR DRUG INDEX gen ( rtcyc rparacet ramox rors riron) (139 differences between tcyc and rtcyc) (131 differences between paracet and rparacet) (113 differences between amox and ramox) (38 differences between ors and rors) (76 differences between iron and riron) STEP 4: GENERATE i7 BY DIVIDING THE VALUES BY 5 TO CONVERT IT TO A PERCENTAGE gen i7 = ( rtcyc+rparacet+ramox+rors+riron)/5 (8 missing values generated) STEP 5: LABELING OF VARIABLES label var drugindex "Drug Availability index (0-5)" STEP 6: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean i7) Un-weighted Scores Province BHC CHC DH Paktya MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN ON TABLE 1.1a Page # 36

38 Weighted Scores Province BHC CHC DH Paktya ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province score Province Score Paktya Family planning availability index Description This indicator measures the availability of family planning supplies in health facilities. The 4 listed items are considered to be sufficient for adequate family planning according to the BPHS. 1. Condoms 2. OCP 3. DMPA 4. IUDs Technical Details Questions # from F7 were used for this indicator. For each of the listed items that were in stock throughout the month prior to the survey, a facility received 1 point. If there was a stock-out in the month prior to the survey the facility receives a score of 0. A simple index is created by adding all the score for each facility. The range of scores possible for each facility is 0-5. A percentage score was calculated out of a maximum of 5 for each facility type within a province and weighted by the distribution of health facilities in the national sample. These weighted scores were then added for each province to obtain the mean for that province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 8 USING STATA TO PRODUCE SCORES FOR INDICATOR 8: FAMILY PLANNING AVAILABILITY INDEX STEP 1: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS gen condom = q608a Page # 37

39 gen ocp = q609a gen dmpa = q610a gen iud= q611a STEP 2: RECODING NOT AVAILABLE AS ZERO 0 AND NO RESPONSES/DONOT KNOW AS MISSING recode condom ocp dmpa iud (2=0) (98/99=.) STEP 3: GENERATING NEW VARIABLE FOR FAMILY PLANNING AVAILABILITY INDEX gen (rcondom rocp rdmpa riud) (22 differences between condom and rcondom) (18 differences between ocp and rocp) (34 differences between dmpa and rdmpa) (60 differences between iud and riud) STEP 4: GENERATE i8 BY DIVIDING THE VALUES BY 3 TO CONVERT IT TO A PERCENTAGE gen i8 = (rcondom+rocp+rdmpa+riud)/4 (5 missing values generated) STEP 5: LABELING OF VARIABLES label i8 "Family Planning Availability index (0-5)" STEP 6: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean i8) Un-weighted Scores Province BHC CHC DH Nangrahar MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN ON TABLE 1.1a Weighted Scores Province BHC CHC DH Nangrahar ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province score Province Score Nangrahar 87.8 Page # 38

40 9. Laboratory functionality index Description This indicator measures the functionality of the facility s laboratory. The facility is not assessed on the basis of whether the facility has a separate laboratory room or a designated laboratory but scores are given on the basis of whether the facility is able to carry out the following 11 tests on the day of the survey. 1. Complete blood counts 2. Malaria smears 3. Rapid diagnostic test for malaria 4. TB smears 5. Gram stains 6. Blood type and cross match 7. Urine dipstick tests 8. HIV testing 9. Liver function testing 10. Syphilis testing 11. Pregnancy testing Technical details Questions # from F7 were used. As per the BPHS guidelines, only CHCs and DHs were included in the analysis of this index. For each of the above tests that a facility was able to perform on the day of the survey, the facility received 1 point. The range of scores possible for each facility is A percentage score was calculated out of a maximum of 11 and weighted by the distribution of health facilities in the national sample. Average scores were calculated for each facility type in a province and were weighted by the distribution of health facilities in the national sample. These weighted scores were then added for each province to obtain the mean for that province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 9 USING STATA TO PRODUCE SCORES FOR INDICATOR 9: LABORATORY FUNCTIONALITY INDEX STEP 1: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS gen cbc = q525 gen mal_smear = q526 gen tb_smear= q527 gen gramstain = q528 gen bloodtype = q529 Page # 39

41 gen hivtest= q530 gen heptest = q531 gen syphtest = q532 gen mal_rapdx= q533 gen urinedip = q534 gen pregtest = q535 STEP 2: RECODING NO RESPONSES AS MISSING FOR ANY ITEM NOT PRESENT OR PRESENT AND NOT FUNCTIONAL, THE FACILITY GETS A SCORE OF 0 recode cbc mal_smear mal_rapdx tb_smear gramstain bloodtype urinedip hivtest heptest syphtest pregtest (1=1) (2=0) (3=0) (98/99=.) gen (rcbc rmal_smear rmal_rapdx rtb_smear rgramstain rbloodtype rurinedip rhivtest rheptest rsyphtest rpregtest) (392 differences between cbc and rcbc) (357 differences between mal_smear and rmal_smear) (517 differences between mal_rapdx and rmal_rapdx) (349 differences between tb_smear and rtb_smear) (515 differences between gramstain and rgramstain) (449 differences between bloodtype and rbloodtype) (386 differences between urinedip and rurinedip) (437 differences between hivtest and rhivtest) (593 differences between heptest and rheptest) (542 differences between syphtest and rsyphtest) (303 differences between pregtest and rpregtest) STEP 3: GENERATING NEW VARIABLE FOR FUNCTIONAL LABORATORY INDEX gen i9 = (rcbc+rmal_smear+rmal_rapdx+rtb_smear+rgramstain+rbloodtype+rurinedip+rhivtest+rheptest+rsy phtest+rpregtest)/11 (14 missing values generated) STEP 4: DIVIDE BY 11 TO CONVERT IT TO A PERCENTAGE Gen labindex = i9/11 STEP 5: LABELING OF VARIABLES label var i9 "Laboratory Functionality Index (0-5)" STEP 6: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND FACILITY TYPE CALCULATED ONLY FOR CHC AND DISTRICT HOSPITALS AND NOT FOR BHC table province fac_type if fac_type!=1, c(mean i9) center Un-weighted Scores Province BHC CHC DH Nangrahar Page # 40

42 MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN ON TABLE 1.1b Weighted Scores Province BHC CHC DH Nangrahar ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province score Province Score Nangrahar Staffing index meeting minimum staff requirements Description This indicator measures whether facilities meet BPHS requirements for total number of clinical staff per facility type. Technical details Clinical staff is defined as a doctor, nurse or midwife. The number of clinical staff required by the BPHS varies with facility type: BHC: CHC: District Hospitals: 2 clinical staff required 6 clinical staff required 21 clinical staff required Questions # 301 to 309 from F7 were used. Each health facility type had a different standard. Each BHCs is given a score of 1 if a maximum of 2 clinical staff is present, CHCs is given a score of 1 if a maximum of 6 clinical staff is present and a DHs is given a score of if a maximum of 21 clinical staff is present. The number of health facilities that had all required clinical staff was converted into a percentage for each facility type. Average scores were calculated for each facility type in a province and were weighted by the distribution of health facilities in the national sample. These weighted scores were then added for each province to obtain the mean for that province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 10 Page # 41

43 USING STATA TO PRODUCE SCORES FOR INDICATOR 10: STAFFING INDEX MEETING MINIMUM STAFF GUIDELINES STEP 1: RECODING NO RESPONSES/DONOT KNOW AS ZEROS 0 recode q303am q303af q303bm q303bf q303cm q303cf q303dm q303df q303em q303ef q303fm q303ff q303gm q303gf q303hm q303hf q303im q303if (98/99=0) (q303am: 5 changes made) (q303af: 6 changes made) (q303bm: 5 changes made) (q303bf: 5 changes made) (q303cm: 6 changes made) (q303cf: 6 changes made) (q303dm: 6 changes made) (q303df: 6 changes made) (q303em: 2 changes made) (q303ef: 2 changes made) (q303fm: 7 changes made) (q303ff: 7 changes made) (q303gm: 2 changes made) (q303gf: 4 changes made) (q303hm: 3 changes made) (q303hf: 1 changes made) (q303im: 8 changes made) (q303if: 7 changes made) STEP 2: GENERATING VARIABLE FOR DOCTORS AND NURSES FOR DOCTORS WHO HAVE WORKED IN THE PAST MONTH gen dr = q303am + q303af + q303bm + q303bf + q303cm + q303cf + q303dm + q303df + q303em + q303ef + q303fm + q303ff label var dr "Total number of doctors who have worked in past month" FOR NURSES WHO HAVE WORKED IN THE PAST MONTH gen nursemw = q303gm + q303gf + q303hm + q303hf + q303im + q303if label var nursemw "Number of nurses, asst. nurses, midwives and aux. midwives worked in past month" TOTAL NUMBER OF DOCTORS AND NURSES WORKED IN THE PAST MONTH gen drnursemw = dr + nursemw label var drnursemw "Total number of doctors, nurses and midwives working in past month" BHC MEETING BPHS REQUIREMENT FOR # OF DR/NURSE/MW gen staff_bhc=0 if drnursemw<2 & fac_type==1 (562 missing values generated) replace staff_bhc=1 if drnursemw>=2 & fac_type==1 Page # 42

44 (329 real changes made) label var staff_bhc "BHC meets BPHS requirement for # of dr/nurse/mw" label values staff_bhc y CHC MEETING BPHS REQUIREMENT FOR # OF DR/NURSE/MW gen staff_chc=0 if drnursemw<6 & fac_type==2 (500 missing values generated) replace staff_chc=1 if drnursemw>=6 & fac_type==2 (72 real changes made) label var staff_chc "CHC meets BPHS requirement for # of dr/nurse/mw" HOSPITAL MEETING BPHS REQUIREMENT FOR # OF DR/NURSE/MW gen staff_hospital=0 if drnursemw<21 & fac_type==3 (591 missing values generated) replace staff_hospital=1 if drnursemw>=21 & fac_type==3 (16 real changes made) label var staff_hospital "Hospital meets BPHS requirement for # of dr/nurse/mw" STEP 3: GENERATING VARIABLE FOR FACILITIES MEETING BPHS REQUIREMENT FOR # OF DR/NURSE/MW FOR FACILITY TYPE gen i10=0 if staff_bhc==0 staff_chc==0 staff_hospital==0 (417 missing values generated) replace i10=1 if staff_bhc==1 staff_chc==1 staff_hospital==1 (417 real changes made) replace i10=. if drnursemw==. (0 real changes made) label var i10 "Meets BPHS requirement for # of dr/nurse/mw for facility type" STEP 4: TABULATING OVERALL SATISFACTION VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean i10) center Un-weighted Scores Province BHC CHC DH Laghman MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN ON TABLE 1.1d Weighted Scores Province BHC CHC DH Page # 43

45 Laghman ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province score Province Score Laghman Provider knowledge score Description Assessment methodology of this indicator was revised In 2008 to measure knowledge of health workers more accurately in priority areas for MoPH. In the revised methodology, clinical case scenarios were presented to the health workers in three areas: 1. Vaccination 2. Integrated Management of Childhood Illness (IMCI) 3. Reproductive health As in previous years this indicator measures the level of knowledge possessed by doctors, assistant doctors, nurses, midwives and auxiliary midwives. In the revised methodology used in 2008, vaccinators were also included in the knowledge assessment.the table belows shows the detailed assessment scheme of different cadres on the basis of the three priority areas. AREA Vaccination IMCI Reproductive Health ASSESSMENT DONE WITH All clinical staff (including vaccinators) Doctors, Assistant doctors, Nurses Midwives, Auxilliary midwives Technical details Questions 199 and 199 A from F5 were used for all clinical staff (including vaccinators) Questions 200 to 220 from F5 include four case scenarios and were for Doctors, Assistant doctors and Nurses Question 221 to 253 from F5 includes 3 case scenarios and RH Practice Questions and were for Midwives and Auxilliary midwives. STATA DEMONSTRATION 11 USING STATA TO PRODUCE SCORES FOR INDICATOR 11: HEALTH WORKER KNOWLEDGE STEP 1: CREATING HEALTH WORKER CATEGORIES FOR THE DIFFERENT SETS OF KNOWLEDGE QUESTIONS GENERATE AND RECODE VARIABLE FOR POSITION OF THE HEALTH WORKER AS Page # 44

46 DESIGNATED BY THE MOPH tab q107 gen q107_new=q107 replace q107_new=5 if q107==8 & q107specify1==12 (5 real changes made) label variable q107_new "Position of HW - recode" label define q107_new 1"Doctor" 2"Nurse" 3"Midwife" 4"Auxillary Midwife" 5"Assistant Doctor" 6"CHS" 7"Vaccinator" 8"Other" label values q107_new q107_new GENERATING HW CATEGORY RELEVANT FOR MEASURING IMMUNIZATION KNOWLEDGE gen hw1=1 if q107_new==1 q107_new==2 q107_new==3 q107_new==4 q107_new==5 q107_new==7 (143 missing values generated) MEASURING KNOWLEDGE ABOUT IMMUNISATIONS (Dichotomize antigens to see what the distribution of right/wrong answers is) **BCG gen BCGyn=0 replace BCGyn=1 if q199a_bcg>=0 & q199a_bcg<=4 (2062 real changes made) label define BCGyn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values BCGyn BCGyn label variable BCGyn "Answer BCG schedule correctly" **OPV0 gen OPV0yn=0 replace OPV0yn=1 if q199b_opv0>=0 & q199b_opv0<=4 (1955 real changes made) label define OPV0yn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values OPV0yn OPV0yn label variable OPV0yn "Answer OPV0 schedule correctly" **OPV1 - are counting +/- 1 month for the dates. For OPV1 the EPI schedule is 6 weeks. +/- 1 month would be between 2 wks and 10 wks. gen OPV1yn=0 replace OPV1yn=1 if q199c_opv1>=2 & q199c_opv1<=10 (1910 real changes made) Page # 45

47 label define OPV1yn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values OPV1yn OPV1yn label variable OPV1yn "Answer OPV1 schedule correctly" **OPV2 - are counting +/- 1 month for the dates. For OPV1 the EPI schedule is 10 weeks. +/- 1 month would be between 6 wks and 14 wks. gen OPV2yn=0 replace OPV2yn=1 if q199d_opv2>=6 & q199d_opv2<=14 (1804 real changes made) label define OPV2yn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values OPV2yn OPV2yn label variable OPV2yn "Answer OPV2 schedule correctly" **OPV3 - are counting +/- 1 month for the dates. For OPV1 the EPI schedule is 14 weeks. +/- 1 month would be between 10 wks and 18 wks. gen OPV3yn=0 replace OPV3yn=1 if q199e_opv3>=10 & q199e_opv3<=18 (1719 real changes made) label define OPV3yn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values OPV3yn OPV3yn label variable OPV3yn "Answer OPV3 schedule correctly" **DPT1 - are counting +/- 1 month for the dates. For DPT1 the EPI schedule is 6 weeks. +/- 1 month would be between 2 wks and 10 wks. gen DPT1yn=0 replace DPT1yn=1 if q199g_dpt_hepb1>=2 & q199g_dpt_hepb1<=10 (1689 real changes made) label define DPT1yn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values DPT1yn DPT1yn label variable DPT1yn "Answer DPT1 schedule correctly" **DPT2 - are counting +/- 1 month for the dates. For DPT2 the EPI schedule is 10 weeks. +/- 1 month would be between 6 wks and 14 wks. gen DPT2yn=0 replace DPT2yn=1 if q199h_dpt_hepb2>=6 & q199h_dpt_hepb2<=14 (1691 real changes made) label define DPT2yn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values DPT2yn DPT2yn label variable DPT2yn "Answer DPT2 schedule correctly" Page # 46

48 **DPT3 - are counting +/- 1 month for the dates. For DPT3 the EPI schedule is 14 weeks. +/- 1 month would be between 10 wks and 18 wks. gen DPT3yn=0 replace DPT3yn=1 if q199i_dpt_hepb3>=10 & q199i_dpt_hepb3<=18 (1652 real changes made) label define DPT3yn 0"NO/DK/Incorrect Answer" 1"Correct Answer" label values DPT3yn DPT3yn label variable DPT3yn "Answer DPT3 schedule correctly" **Measles1 - are counting +/- 1 month for the dates. For measles1 vaccine the EPI schedule is 9 months (36 wks). +/- 1 month would be between 32 wks and 40 wks. gen measlesyn=0 replace measlesyn=1 if q199j_measles1_vaccine>=32 & q199j_measles1_vaccine<=40 (1920 real changes made) label define measlesyn 0"NO/DK/Incorrect Answer" 1"Correct Answer", modify label values measlesyn measlesyn label variable measlesyn "Answer measles schedule correctly" **What percentage of providers answered the whole EPI schedule correctly gen EPI_full=0 replace EPI_full=1 if BCGyn==1 & OPV0yn==1 & OPV1yn==1 & OPV2yn==1 & OPV3yn==1 & DPT1yn==1 & DPT2yn==1 & DPT3yn==1 & measlesyn==1 (1458 real changes made) replace EPI_full=. if hw1!=1 (143 real changes made, 143 to missing) label define EPI_full 0"NO/DK/Incorrect Answer" 1"Correct Answer", modify label values EPI_full EPI_full label variable EPI_full "Answer the full EPI schedule correctly" tab EPI_full **What to do about vaccination in a given situation - A scenario Q199aa-Q199ad **Recoding q199aa-ad recode q199aa q199ab q199ac q199ad (2=0)(1=1) (98 99 =.), gen (q199aa_new q199ab_newq199ac_new q199ad_new) (1937 differences between q199aa and q199aa_new) (425 differences between q199ab and q199ab_new) (2203 differences between q199ac and q199ac_new) (2185 differences between q199ad and q199ad_new) **Count number of answers people gave to this scenario Page # 47

49 gen q199a_d= q199aa_new+q199ab_new+q199ac_new+q199ad_new (120 missing values generated) replace q199a_d=0 if hw1==1 & q199a_d==. (32 real changes made) replace q199a_d=. if hw1!=1 (55 real changes made, 55 to missing) label define q199a_d 0"Gave no ans" 1"Gave one ans" 2"Gave two ans", modify label values q199a_d q199a_d label variable q199a_d "# of answers given by HWs - vacc scenario" tab q199a_d **If health workers who are supposed to be interviewed for this answer do not know it or there is missing value, it is the same as not getting the right answer. That is why I coded them as "0" **Answers q199ab_new and q199ac_new are acceptable but q199ab has to be part of the answer. gen vacc_scene=0 if q199a_d==0 (2201 missing values generated) replace vacc_scene=0 if q199ab_new==0 (305 real changes made) replace vacc_scene=1 if q199a_d>=1 & q199ab_new==1 (1808 real changes made) replace vacc_scene=0 if hw1==1 & vacc_scene==. (0 real changes made) replace vacc_scene=. if hw1!=1 (55 real changes made, 55 to missing) label define vacc_scene 0"Wrong answer" 1"Right answer" label values vacc_scene vacc_scene label variable vacc_scene "Knowledge of vaccine scenario" tab vacc_scene **Vaccine Knowledge gen vacc_know_index=(epi_full+vacc_scene)/2 (143 missing values generated) tab vacc_know_index sum vacc_know_index Page # 48

50 **IMCI KNOWLEDGE INDEX **This index is only applicable to doctors, nurses, assistant doctors gen hw2=1 if q107_new==1 q107_new==2 q107_new==5 (1273 missing values generated) **RECODE THE IMCI DISEASE IDENTIFICATION QUESTIONS recode q201 q206 q211 q216 (98 99=.), gen(q201_new q206_new q211_new q216_new) (3 differences between q201 and q201_new) (3 differences between q206 and q206_new) (4 differences between q211 and q211_new) (3 differences between q216 and q216_new **IMCI SCENARIO 1 **Disease ID gen imci1_id=0 if q201_new!=3 (824 missing values generated).replace imci1_id=1 if q201_new==3 (824 real changes made) replace imci1_id=1 if q201_new==4 & (q201_specify1==2 q201_specify1==7 q201_specify1= > =14) (60 real changes made) replace imci1_id=. if hw2!=1 (1273 real changes made, 1273 to missing) label define imci1_id 0"Incorrect Ans" 1"Correct Ans", modify label values imci1_id imci1_id label variable imci1_id "Disease id IMCI Scenario 1" tab imci1_id **Treatment identification **Acceptable answers are 204a,b,c,d,f. In other category 12, 14, 19, 20. recode q204a q204b q204c q204d q204f (2=0)(98 99 =.), gen(q204a_new q204b_new q204c_new q204d_new q204f_new) (635 differences between q204a and q204a_new) (424 differences between q204b and q204b_new) (416 differences between q204c and q204c_new) (639 differences between q204d and q204d_new) (723 differences between q204f and q204f_new) recode q204o_other (2=0) (98 99 =.), gen (q204o_new) (637 differences between q204o_other and q204o_new) label define ny 0"No" 1"Yes" Page # 49

51 label values q204a_new ny label values q204b_new ny label values q204c_new ny label values q204d_new ny label values q204f_new ny replace q204a_new=1 if q204o_specify1==19 (5 real changes made) replace q204b_new=1 if q204o_specify1==14 (1 real change made) replace q204d_new=1 if q204o_specify1==12 (1 real change made) replace q204o_new=0 if q204o_specify1!=20 (1602 real changes made) label values q204o_new ny **Giving credit to provider if they mentioned giving fluids (b, c, d, f) as part of their answer and/or hospitalization** gen imci1_treat=0 if q204a_new==0 & q204b_new==0 & q204c_new==0 & q204d_new==0 & q204f_new==0 & q204o_new==0 (2183 missing values generated) replace imci1_treat=1 if q204a_new==1 q204b_new==1 q204c_new==1 q204d_new==1 q204f_new==1 q204o_new==1 (767 real changes made) replace imci1_treat=0 if q204o_new==0 & q204a_new==. & q204b_new==.& q204c_new==.& q204d_new==. & q204f_new==. (1416 real changes made) *Giving credit for correct treatment answers by respondents who initially gave wrong answer of malaria gen imci1_mal = 0 if hw2==1 (1273 missing values generated) recode q202a q202b q202c (2=0)(98 99 =.), gen(q202a_new q202b_new q202c_new) (5 differences between q202a and q202a_new) (4 differences between q202b and q202b_new) (5 differences between q202c and q202c_new) label values q202a_new ny label values q202b_new ny label values q202c_new ny tab q202n_specify1 replace imci1_mal = 1 if (q202a_new==1 q202b_new==1 q202c_new==1) (1 real change made) Page # 50

52 label var imci1_mal "Correct diarrhea treatment among incorrectly diagnosed malaria" tab imci1_mal replace imci1_treat=1 if imci1_mal==1 (1 real change made) *Giving credit for correct treatment answers by respondents who initially gave wrong answer of infection gen imci1_inf = 0 if hw2==1 (1273 missing values generated). recode q203a q203b (2=0)(98 99 =.), gen(q203a_new q203b_new ) (11 differences between q203a and q203a_new) (14 differences between q203b and q203b_new) label values q203a_new ny label values q203b_new ny tab q203o_specifya replace imci1_inf = 1 if (q203a_new==1 q203b_new==1) (11 real changes made) label var imci1_inf "Correct diarrhea treatment among incorrectly diagnosed infection" tab imci1_inf replace imci1_treat=1 if imci1_inf==1 (11 real changes made) *Giving credit for correct treatment answers by respondents who initially gave a correct "other" response for disease ID recode q205a q205b q205c q205d q205f (2=0)(98 99 =.), gen(q205a_new q205b_new q205c_new q205d_new q205f_new) (81 differences between q205a and q205a_new) (50 differences between q205b and q205b_new) (94 differences between q205c and q205c_new) (77 differences between q205d and q205d_new) (93 differences between q205f and q205f_new) tab1 q205a_new -q205f_new label values q205a_new ny label values q205b_new ny label values q205c_new ny label values q205d_new ny label values q205f_new ny tab q205s tab q205_specify1 gen imci1_treatoth = 0 if hw2==1 Page # 51

53 (1273 missing values generated) gen imci1_validoth = 1 if hw2==1&(q201_new==4 & (q201_specify1==2 q201_specify1==7 q201_specify1==14)) (2173 missing values generated) replace imci1_treatoth = 1 if imci1_validoth==1& (q205a_new==1 q205b_new==1 q205cnew==1 q205d_new==1 q205f_new==1 q205_specify1==4) (58 real changes made). replace imci1_treat = 1 if imci1_treatoth==1 (58 real changes made) replace imci1_treat =. if hw2!=1 (1273 real changes made, 1273 to missing) *We will take out this stipulation b/c scoring is now disaggregated; /// Treatment will be scored independently of disease identification *replace imci1_treat=0 if imci1_id==0 tab imci1_treat if hw2==1 label values imci1_treat ny label variable imci1_treat "Correct treatment for diarrhea scenario" tab imci1_treat if hw2==1 **IMCI SCENARIO 2 **Disease ID gen imci2_id=0 if q206_new!=2 (200 missing values generated) replace imci2_id=1 if q206_new==2 (200 real changes made) replace imci2_id=1 if q206_new==4 & (q206_specify1==1 q206_specify1==2 q206_specify1==3 q206_specify1==7 q206_specify2==1 q2 06_specify2==2 q206_specify2==3) (722 real changes made) replace imci2_id=. if hw2!=1 (1273 real changes made, 1273 to missing) label define imci2_id 0"Incorrect Ans" 1"Correct Ans", modify label values imci2_id imci2_id label variable imci2_id "Disease id IMCI Scenario 2" tab imci2_id **Treatment identification Page # 52

54 **Acceptable answers are a, d, f, g. In other category, recode cotrimoxale recode q208a q208d q208f q208g (2=0)(98 99 =.), gen(q208a_new q208d_new q208f_new q208g_new) (145 differences between q208a and q208a_new) (149 differences between q208d and q208d_new) (171 differences between q208f and q208f_new) (92 differences between q208g and q208g_new) gen q208o_new=0 replace q208o_new=1 if q208_other==1 & (q208_specifya==2 q208_specifya==8 q208_specifyb==2) (2 real changes made) label values q208o_new ny **Giving credit to provider if they mentioned giving antibiotics (a,d, f, g)) as part of their answer and referring the child to the hospital(a). ** gen imci2_treat=0 if q208a_new==0 & q208d_new==0 & q208f_new==0 & q208g_new==0 & q208o_new==0 (2228 missing values generated) replace imci2_treat=1 if q208a_new==1 q208d_new==1 q208f_new==1 q208g_new==1 q208o_new==1 (190 real changes made) replace imci2_treat=0 if q208o_new==0 & q208a_new==. & q208d_new==. & q208f_new==. &q208g_new==. (2038 real changes made) *Removing this stipulation b/c diagnosis and treatment will now be scored separately *replace imci2_treat=0 if imci2_id==0 replace imci2_treat=. if imci2_id==. (1273 real changes made, 1273 to missing) **Giving credit for correct treatment answers by respondents who initially gave incorrect diagnosis of malaria recode q207a q207d q207e (2=0)(98 99 =.), gen(q207a_new q207d_new q207e_new) (4 differences between q207a and q207a_new) (4 differences between q207d and q207d_new) (2 differences between q207e and q207e_new) label values q207a_new ny label values q207d_new ny label values q207e_new ny tab q207n_specifya replace imci2_treat=1 if Page # 53

55 (q207a_new==1 q207d_new==1 q207e_new==1 q207n_specifya==2)&hw2==1 (2 real changes made) **Giving credit for correct treatment answers by respondents who initially gave incorrect diagnosis of diarrhea recode q209a q209f q209h q209i (2=0)(98 99 =.), gen(q209a_new q209f_new q209h_new q209i_new) (1 differences between q209a and q209a_new) (1 differences between q209f and q209f_new) (1 differences between q209h and q209h_new) (1 differences between q209i and q209i_new) tab q209_specifya replace imci2_treat=1 if (q209a_new==1 q209f_new==1 q209h_new==1 q209i_new==1)&hw2==1 (0 real changes made) **Now scoring the other treatment responses if a HW gave a valid "Other" response gen imci2_treatoth = 0 if hw2==1 (1273 missing values generated) replace imci2_treatoth = 1 if q206_new==4 & (q206_specify1==1 q206_specify1==2 q206_specify1==3 q206_specify1==7 q206_specify2==1 q2 06_specify2==2 q206_specify2==3) (722 real changes made) recode q210a q210f q210h q210i(2=0)(98 99 =.) if imci2_treatoth==1, gen( q210a_new q210f_new q210h_new q210i_new) (482 differences between q210a and q210a_new) (495 differences between q210f and q210f_new) (627 differences between q210h and q210h_new) (419 differences between q210i and q210i_new) label values q210a_new ny label values q210f_new ny label values q210h_new ny label values q210i_new ny tab q210_specifya *Creating a correct category if correct responses were mentioned in the "other" response gen q210o_new = 0 if hw2==1& imci2_treatoth ==1 (1514 missing values generated) replace q210o_new =1 if (q210_specifya==2 q210_specifya==12) (36 real changes made) *Giving credit to providers who initially mentioned a correct other response replace imci2_treat = 1 if (q210a_new==1 q210f_new==1 q210h_new==1 q210i_new==1 q210o new==1) (667 real changes made) Page # 54

56 replace imci2_treat=. if hw2!=1 (3 real changes made, 3 to missing) tab imci2_treat label values imci2_treat ny label variable imci2_treat "Correct treatment for infection scenario 2" tab imci2_treat **IMCI SCENARIO 3 **Disease ID gen imci3_id=0 if q211_new!=2&hw2==1 (1399 missing values generated) replace imci3_id=1 if q211_new==2 (126 real changes made) replace imci3_id=1 if q211_new==4 & (q211_specifya==1 q211_specifya==10 q211_specifyb==1) (679 real changes made) replace imci3_id=. if hw2!=1 (3 real changes made, 3 to missing) label define imci3_id 0"Incorrect Ans" 1"Correct Ans", modify label values imci3_id imci3_id label variable imci3_id "Disease id IMCI Scenario 3" tab imci3_id **Treatment identification **Acceptable answers are a, d, e, f, g. In other category, 1 and 21 recode q213a q213d q213e q213f q213g (2=0)(98 99 =.), gen(q213a_new q213d_new q213e_new q213f_new q213g_new) (86 differences between q213a and q213a_new) (78 differences between q213d and q213d_new) (113 differences between q213e and q213e_new) (97 differences between q213f and q213f_new) (76 differences between q213g and q213g_new) label values q213a_new ny label values q213d_new ny label values q213f_new ny label values q213g_new ny gen q213o_new=0 Page # 55

57 replace q213o_new=1 if q213_other==1 & (q213_specifya==1 q213_specifya==21) (1 real change made) label values q213o_new ny **Giving credit to provider if they mentioned giving antibiotics (a,d, e, f, g)) as part of their answer and referring the child to the hospital(a). ** gen imci3_treat=0 if q213a_new==0 & q213d_new==0 & q213e_new==0 & q213f_new==0 & q213g_new==0 & q213o_new==0 (2229 missing values generated) replace imci3_treat=1 if q213a_new==1 q213d_new==1 q213e_new==1 q213f_new==1 q213g_new==1 q213o_new==1 (118 real changes made) replace imci3_treat=0 if q213o_new==0 & q213a_new==. & q213d_new==. & q213f_new==. & q213g_new==. (2111 real changes made) *Removing this stipulation because we are now giving credit separately for treatment and diagnosis *replace imci3_treat=0 if imci3_id==0 replace imci3_treat=. if imci3_id==. (1273 real changes made, 1273 to missing) replace imci3_treat=. if hw2!=1 (0 real changes made) tab imci3_treat label values imci3_treat ny label variable imci3_treat "Correct treatment for infection scenario" **Giving credit for correct treatment answers by respondents who initially gave incorrect diagnosis of malaria recode q212a q212d q212e (2=0)(98 99 =.), gen(q212a_new q212d_new q212e_new) (2 differences between q212a and q212a_new) (2 differences between q212d and q212d_new) (1 differences between q212e and q212e_new) tab q212_specifya replace imci3_treat=1 if (q212a_new==1 q212d_new==1 q212e_new==1) &hw2==1 (1 real change made) *Giving credit if a valid "other" diagnosis was given and correct treatment options given gen imci3_treatoth=0 if hw2==1 (1273 missing values generated) Page # 56

58 replace imci3_treatoth = 1 if q211_new==4 & (q211_specifya==1 q211_specifya==10 q211 specifyb==1) (679 real changes made). recode q215a q215f q215g q215h q215i(2=0)(98 99 =.) if imci3_treatoth==1, gen( q215anew q215f_new q215g_new q215h_new q215i_new) (481 differences between q215a and q215a_new) (475 differences between q215f and q215f_new) (659 differences between q215g and q215g_new) (515 differences between q215h and q215h_new) (402 differences between q215i and q215i_new) label values q215a_new ny label values q215f_new ny label values q215g_new ny label values q215h_new ny label values q215i_new ny *Creating an other variable if antibiotics or cotrimoxizole were mentioned in the other category gen q215o_new=0 if imci3_treatoth==1 (1554 missing values generated) replace q215o_new=1 if imci3_treatoth==1& q215_other==1 & (q215_specify1==12 q215_specify1==16) (30 real changes made) label values q215o_new ny *Giving credit for these valid responses even if other diagonsis was mentioned replace imci3_treat = 1 if (q215a_new==1 q215f_new==1 q215g_new==1 q215h_new==1 q215inew==1 q215o_new==1) (665 real changes made) replace imci3_treat=. if hw2!=1 (3 real changes made, 3 to missing) tab imci3_treat **IMCI SCENARIO 4 - Dropping Scenario 4 *Generating a 6-point IMCI knowledge index, giving separate credit for diagnosis and treatment for each of the three scenarios gen imci_index = (imci1_id + imci1_treat + imci2_id + imci2_treat + imci3_id + imci3_treat)/6 (1273 missing values generated) Page # 57

59 REPRODUCTIVE HEALTH KNOWLEDGE **HW category relevant for measuring RH KNOWLEDGE gen hw3=1 if q107_new==3 q107_new==4 (1740 missing values generated) label variable hw3 "Midwives and auxillary midwives" **Case scenario 5 gen q222_new=q222 (1736 missing values generated) replace q222_new=. if q222==99 (2 real changes made, 2 to missing) **Disease ID gen RH1_id=0 replace RH1_id=1 if q222_new==1 (239 real changes made) replace RH1_id=1 if q222_new==6 & (q222specify1==1 q222specify1==3 q222specify1==7) (48 real changes made) replace RH1_id=0 if q222_new==7 (0 real changes made) replace RH1_id=. if hw3!=1 (1740 real changes made, 1740 to missing) label define RH1_id 0"Incorrect Ans" 1"Correct Ans", modify label values RH1_id RH1_id label variable RH1_id "Disease id RH Scenario 1" **Treatment identification **Acceptable answers are a, b, c, d, f and e (if any one of the others are present but not by itself). In other category, 2,4,5,7,11,12,13,17,20, and 22. recode q223a q223b q223c q223d q223e q223f (2=0)(98 99 =.), gen(q223a_new q223b_new q223c_new q223d_new q223e_new q223f_new) (361 differences between q223a and q223a_new) (226 differences between q223b and q223b_new) (339 differences between q223c and q223c_new) (141 differences between q223d and q223d_new) (334 differences between q223e and q223e_new) (329 differences between q223f and q223f_new) Page # 58

60 label values q223a_new ny label values q223b_new ny label values q223c_new ny label values q223d_new ny label values q223e_new ny label values q223f_new ny gen q223g_new=0 replace q223g_new=1 if q223g_other==1 & (q223g_specify1==2 q223g_specify1==4 q223g_specify1==5 q223g_specify1==7 q223g_specify1= =11 q223g_specify1==12 q223g_specify1==13 q223g_specify1==17 q223g_specify1==20 q223g_s pecify1==22) (61 real changes made) label values q223g_new ny **Giving credit to provider if they mentioned a, b, c, d, f and g as part of their answer and if they mentioned in e in conjunction with a, b,c, d, f. ** gen RH1_treat=0 if q223a_new==0 & q223b_new==0 & q223c_new==0 & q223d_new==0 & q223e_new==0 & q223f_new==0 & q223g_new==0 (2207 missing values generated) replace RH1_treat=1 if q223a_new==1 q223b_new==1 q223c_new==1 q223d_new==1 q223f_new==1 q223g_new==1 (459 real changes made) replace RH1_treat=1 if q223e_new==1 & (q223a_new==1 q223b_new==1 q223c_new==1 q223d_new==1 q223f_new==1 q223g_new==1) (0 real changes made) replace RH1_treat=0 if q223e_new==1 & q223a_new==0 & q223b_new==0 & q223c_new==0 & q223d_new==0 & q223f_new==0 & q223g_new==0 (7 real changes made) replace RH1_treat=0 if q223g_new==0 & q223a_new==. & q223b_new==. & q223c_new==. & q223d_new==. & q223e_new==. & q223f_new==. (1741 real changes made) *Removing the following stipulation b/c credit will be given for treatment and diagnosis separately (replace RH1_treat=0 if RH1_id==0) replace RH1_treat=. if RH1_id==. (1740 real changes made, 1740 to missing) label values RH1_treat ny label variable RH1_treat "Correct treatment for pre-eclampsia scenario" **Case scenario 6 **Treatment identification Page # 59

61 **Having two level scoring. One for the drying warming part (any combination of a, c and 14 in the other category)and another for clearing airway (any combination of e and 1, 3, 4, 5, 8 in the other category. ** recode q224a q224b q224c q224d q224e q224g (2=0)(98 99 =.), gen(q224a_new q224b_new q224c_new q224d_new q224e_new q224g_new ) (192 differences between q224a and q224a_new) (400 differences between q224b and q224b_new) (303 differences between q224c and q224c_new) (264 differences between q224d and q224d_new) (111 differences between q224e and q224e_new) (494 differences between q224g and q224g_new) label values q224a_new ny label values q224b_new ny label values q224c_new ny label values q224d_new ny label values q224e_new ny label values q224g_new ny gen q224f_new1=0 replace q224f_new1=1 if q224f_other==1 & (q224f_specify1==14) (2 real changes made) label values q224f_new1 ny label variable q224f_new1 "Drying/warming" gen q224f_new2=0 replace q224f_new2=1 if q224f_other==1 & (q224f_specify1==1 q224f_specify1==3 q224f_specify1==4 q224f_specify1==5 q224f_specify1==8 q224f_specify2==1 q224f_specify2==3 q224f_specify2==8) (190 real changes made) label values q224f_new2 ny label variable q224f_new1 "Clearing airways" **Giving credit to provider for mentioning drying/warming (a, c) and if they had 1 in q224f_new1 (or 14 in q224f_other). **/ gen RH2_treat1=0 if q224a_new==0 & q224c_new==0 & q224f_new1==0 (2095 missing values generated) replace RH2_treat1=1 if q224a_new==1 q224c_new==1 q224f_new1==1 (357 real changes made) replace RH2_treat1=0 if q224f_new1==0 & q224a_new==. & q224c_new==. (1738 real changes made) replace RH2_treat1=. if hw3!=1 (1740 real changes made, 1740 to missing) replace RH2_treat1=0 if hw3==1 & q224a_new==. & q224c_new==. & q224f_new1==. Page # 60

62 (0 real changes made) label values RH2_treat1 ny label variable RH2_treat1 "Keeping baby warm/dry for fetal distress" **Giving credit to provider for mentioning ventilation (e) ONLY if they had 1 in q224f_new2 (or 1, 3, 4, 5, 8 in q224f_other). ** *This is one of the two "lumped" RH questions, as keeping baby warm/dry and ventilating are critical steps that should go together gen RH2_treat2=0 if q224e_new==0 & q224f_new2==0 (2168 missing values generated) replace RH2_treat2=1 if q224e_new==1 q224f_new2==1 (430 real changes made) replace RH2_treat2=0 if q224f_new2==0 & q224e_new==. (1738 real changes made) replace RH2_treat2=. if hw3!=1 (1740 real changes made, 1740 to missing) replace RH2_treat2=0 if hw3==1 & q224e_new==. & q224f_new2==. (0 real changes made) replace RH2_treat2=0 if RH2_treat1==0 (123 real changes made) label values RH2_treat2 ny label variable RH2_treat2 "Ventilation for fetal distress" **Treatment identification **Acceptable answers are a - e. In other category, 5, 8, 9, 10, 15. recode q225a q225b q225c q225d q225e q225g (2=0)(98 99 =.), gen(q225a_new q225b_new q225c_new q225d_new q225e_new q225g_new ) (250 differences between q225a and q225a_new) (297 differences between q225b and q225b_new) (226 differences between q225c and q225c_new) (374 differences between q225d and q225d_new) (397 differences between q225e and q225e_new) (477 differences between q225g and q225g_new) label values q225a_new ny label values q225b_new ny label values q225c_new ny label values q225d_new ny label values q225e_new ny label values q225g_new ny Page # 61

63 gen q225f_new=0 replace q225f_new=1 if q225f_other==1 & (q225f_specify1==5 q225f_specify1==8 q225f_specify1==9 q225f_specify1==10 q225f_specify1== 15 q225f_specify2==8) (46 real changes made) label values q225f_new ny **Giving credit to provider if they mentioned a - e as part of their answer or had value 1 in q225f_new. ** gen RH3_treat=0 if q225a_new==0 & q225b_new==0 & q225c_new==0 & q225d_new==0 & q225e_new==0 & q225f_new==0 (2182 missing values generated) replace RH3_treat=1 if q225a_new==1 q225b_new==1 q225c_new==1 q225d_new==1 q225e_new==1 q225f_new==1 (426 real changes made) replace RH3_treat=0 if q225f_new==0 & q225a_new==. & q225b_new==. & q225c_new==. & q225d_new==. & q225e_new==. & q225g_new==. (1739 real changes made) replace RH3_treat=. if hw3!=1 (1740 real changes made, 1740 to missing) replace RH3_treat=0 if hw3==1 & RH3_treat==. (17 real changes made) label values RH3_treat ny label variable RH3_treat "Correct treatment for fetal distress - 3" **Treatment identification **Giving credit to provider if they mentioned 1 or mentioned 1, 10, 17 in the other category. * gen q226_new=q226 (1736 missing values generated) replace q226_new=. if q222==98 q222==99 (2 real changes made, 2 to missing) gen RH4_treat=0 replace RH4_treat=1 if q226_new==1 (338 real changes made) replace RH4_treat=1 if q226_new==2 & (q226_specify1==1 q226_specify1==10 q226_specify1==17 q226_specify2==10) (24 real changes made) replace RH4_treat=0 if q226_new==3 Page # 62

64 (0 real changes made) replace RH4_treat=. if hw3!=1 (1740 real changes made, 1740 to missing) label values RH4_treat ny label variable RH4_treat "Correct treatment for fetal distress - 2" **Treatment identification **Giving credit to provider if they mentioned a - c as part of their answer or had value 3 in q227d_new. If only 9 was mentioned in q227d_new than do not give any credit** recode q227a q227b q227c q227e (2=0)(98 99 =.), gen(q227a_new q227b_new q227c_new q227_new) (262 differences between q227a and q227a_new) (319 differences between q227b and q227b_new) (228 differences between q227c and q227c_new) (471 differences between q227e and q227e_new) label values q227a_new ny label values q227b_new ny label values q227c_new ny label values q227e_new ny gen q227d_new=0 replace q227d_new=1 if q227d_other==1 & (q227d_specify1==3) (8 real changes made) label values q227d_new ny gen RH5_treat=0 if q227a_new==0 & q227b_new==0 & q227c_new==0 & q227d_new==0 (2123 missing values generated) replace RH5_treat=1 if q227a_new==1 q227b_new==1 q227c_new==1 q227d_new==1 (383 real changes made) replace RH5_treat=0 if q227d_new==0 & q227a_new==. & q227b_new==. & q227c_new==. (1740 real changes made) replace RH5_treat=0 if q227d_specify1==9 & q227a_new==0 & q227b_new==0 & q227c_new==0 (0 real changes made) replace RH5_treat=. if hw3!=1 (1740 real changes made, 1740 to missing) replace RH5_treat=0 if hw3==1 & RH5_treat==. (0 real changes made) label define RH5_treat 0"Incorrect Ans" 1"Correct Ans", modify label values RH5_treat RH5_treat label variable RH5_treat "Correct treatment for fetal distress - 3" Page # 63

65 **Case scenario 7 **Treatment identification **Combine questions 229 and 231. While 229 seems to be the diagnostic part and 231 the therapeutic part, answers seem to overlap **q229 - Having two level scoring. One for the diagnostic part (any combination of a - g and 5, 6, 7, 12, 16 in the other category)and another for therapeutic part (any combination of 3, 4, 9, 10 in the other category). **q231 - Diagnostic part** **q229 recode q229a q229b q229c q229d q229e q229f q229g (2=0)(98 99 =.), gen(q229a_new q229b_new q229c_new q229d_new q229e_new q229f_new q229g_new) (353 differences between q229a and q229a_new) (297 differences between q229b and q229b_new) (172 differences between q229c and q229c_new) (96 differences between q229d and q229d_new) (256 differences between q229e and q229e_new) (291 differences between q229f and q229f_new) (337 differences between q229g and q229g_new) label values q229a_new ny label values q229b_new ny label values q229c_new ny label values q229d_new ny label values q229e_new ny label values q229f_new ny label values q229g_new ny gen q229h_new1=0 replace q229h_new1=1 if q229h_other==1 & (q229_specify1==5 q229_specify1==6 q229_specify1==7 q229_specify1==12 q229_specify1==16 q229_specify2==7) (111 real changes made) label values q229h_new1 ny label variable q229h_new1 "Diagnostic" gen q229h_new2=0 replace q229h_new2=1 if q229h_other==1 & (q229_specify1==3 q229_specify1==4 q229_specify1==9 q229_specify1==10) (61 real changes made) label values q229h_new2 ny label variable q229h_new2 "Therapeutic" Page # 64

66 **q231 recode q231a q231b q231c q231e (2=0) (98 99=.), gen(q231a_new q231b_new q231c_new q231e_new) (291 differences between q231a and q231a_new) (34 differences between q231b and q231b_new) (288 differences between q231c and q231c_new) (490 differences between q231e and q231e_new) label values q231a_new ny label values q231b_new ny label values q231c_new ny label values q231e_new ny gen q231d_new1=0 replace q231d_new1=1 if q231d_other==1 & (q231d_specify1==3 q231d_specify1==10 q231d_specify1==11 q231d_specify1==18) (25 real changes made) label values q231d_new1 ny label variable q231d_new1 "Diagnostic" gen q231d_new2=0 replace q231d_new2=1 if q231d_other==1 & (q231d_specify1==4 q231d_specify1==6 q231d_specify1==14 q231d_specify2==6) (19 real changes made) label values q231d_new2 ny label variable q231d_new2 "Therapeutic" **In q229- Giving credit to provider for mentioning diagnostic part (a - g) and if they had 1 in q229h_new1 (or 5, 6, 7, 12, 16 in q229h_other/specify). In q231 - giving credit for diagnostic part if they have 1 in q231d_new1 (or if they got 3, 10, 11, 18 in q231d_ohter/specify) ** gen RH6_treat1=0 if q229a_new==0 & q229b_new==0 & q229c_new==0 & q229d_new==0 & q229e_new==0 & q229f_new==0 & q229g_new==0 & q229h_new1==0 & q231d_new1==0 (2205 missing values generated) replace RH6_treat1=1 if q229a_new==1 q229b_new==1 q229c_new==1 q229d_new==1 q229e_new==1 q229f_new==1 q229g_new==1 q229h_new1==1 q231d_new1==1 (467 real changes made) replace RH6_treat1=0 if q229h_new1==0 & q229a_new==. & q229b_new==. & q229c_new==. & q229d_new==. & q229e_new==. & q229f_new==. & q229g_new==. (1738 real changes made) replace RH6_treat1=. if hw3!=1 (1740 real changes made, 1740 to missing) Page # 65

67 replace RH6_treat1=0 if hw3==1 & q229a_new==. & q229b_new==. & q229c_new==. & q229d_new==. & q229e_new==. & q229f_new==. & q229g_new==. & q229h_new1==. & q231d_new1==. (0 real changes made) label values RH6_treat1 ny label variable RH6_treat1 "Diagnostic for PPH/shock" **Giving credit to provider for mentioning therapeutic part if they had 1 in q229h_new2 (or 3, 4, 9, 10 in q229h_other/specify Or, giving credit for a, b, c in q231a-e or if they got in 1 q231d_new2 (or 4, 6, 14 in q231d_other/specify) ** **Making RH6_treat2 (THerapeutic for PPH/Shock) dependent on correct diagnostic; this is the second of the two "lumped" RH questions gen RH6_treat2=0 if q229h_new2==0 & q231a_new==0 & q231b_new==0 & q231c_new==0 & q231d_new2==0 (2222 missing values generated) replace RH6_treat2=1 if q229h_new2==1 q231a_new==1 q231b_new==1 q231c_new==1 q231d_new2==1 (484 real changes made) replace RH6_treat2=. if hw3!=1 (5 real changes made, 5 to missing) replace RH6_treat2=0 if hw3==1 & q229h_new2==. & q231a_new==. & q231b_new==. & q231c_new==. & q231d_new2==. (0 real changes made) replace RH6_treat2=0 if hw3==1 & RH6_treat2==. (3 real changes made) replace RH6_treat2=0 if RH6_treat1==0 (25 real changes made) label values RH6_treat2 ny label variable RH6_treat2 "Therapeutic for PPH/shock" **Disease ID gen q230_new=q230 (1736 missing values generated) replace q230_new=. if q230==99 (2 real changes made, 2 to missing). gen RH6_id=0 replace RH6_id=1 if q230_new==3 q230_new==5 q230_specify1==3 q230_specify1==6 (436 real changes made) Page # 66

68 replace RH6_id=0 if q230_new==7 (0 real changes made) replace RH6_id=. if hw3!=1 (1740 real changes made, 1740 to missing) label define RH6_id 0"Incorrect Ans" 1"Correct Ans", modify label values RH6_id RH6_id label variable RH6_id "Disease id RH Scenario 7" **Penalizing health workers who while giving the correct treatment gave an incorrect diagnosis. *Removing the following stipulation because we are now giving credit separately for diagnosis and treatment *replace RH6_treat1=0 if RH6_id==0 & RH6_treat1==1 *replace RH6_treat2=0 if RH6_id==0 & RH6_treat2==1 **Treatment - q233 recode q233a q233b q233c q233d q233e_other q233f (2=0) (98 99=.), gen(q233a_new q233b_new q233c_new q233d_new q233e_new q233f_new) (481 differences between q233a and q233a_new) (97 differences between q233b and q233b_new) (158 differences between q233c and q233c_new) (364 differences between q233d and q233d_new) (461 differences between q233e_other and q233e_new) (470 differences between q233f and q233f_new) label values q233a_new ny label values q233b_new ny label values q233c_new ny label values q233d_new ny label values q233e_new ny label values q233f_new ny replace q233a_new=0 if q233b_new==1 q233c_new==1 q233d_new==1 (4 real changes made) **Giving credit to provider for mentioning b or c or d ** gen RH8_treat=0 if q233b_new==0 & q233c_new==0 & q233d_new==0 (2189 missing values generated) replace RH8_treat=0 if q233a_new==1 (0 real changes made) replace RH8_treat=0 if q233e_new==0 q233e_new==1 (451 real changes made) replace RH8_treat=1 if q233b_new==1 q233c_new==1 q233d_new==1 Page # 67

69 (451 real changes made) replace RH8_treat=. if hw3!=1 (5 real changes made, 5 to missing) replace RH8_treat=0 if hw3==1 & q233a_new==. & q233b_new==. & q233c_new==. & q233d_new==. (3 real changes made) replace RH8_treat=0 if hw3==1 & RH8_treat==. (0 real changes made) label values RH8_treat ny label variable RH8_treat "Blood work for PPH/shock" **How to define shock - q234 recode q234a q234b q234c q234d q234e q234g (2=0) (98 99=.), gen(q234a_new q234b_new q234c_new q234d_new q234e_new q234g_new) (247 differences between q234a and q234a_new) (94 differences between q234b and q234b_new) (81 differences between q234c and q234c_new) (305 differences between q234d and q234d_new) (272 differences between q234e and q234e_new) (491 differences between q234g and q234g_new) label values q234a_new ny label values q234b_new ny label values q234c_new ny label values q234d_new ny label values q234e_new ny label values q234g_new ny gen q234f_new=0 replace q234f_new=1 if q234f_other==1 & (q234_specify1==1 q234_specify1==7 q234_specify1==12 q234_specify1==14 q234_specify1==15) (14 real changes made) label values q234f_new ny label variable q234f_new "Diagnostic" **Giving credit to provider credit for a - e and 1, 7, 12, 14, and 15 in other category** gen RH9_treat=0 if q234a_new==0 & q234b_new==0 & q234c_new==0 & q234d_new==0 & q234e_new==0 & q234f_new==0 (2224 missing values generated) replace RH9_treat=1 if q234a_new==1 q234b_new==1 q234c_new==1 q234d_new==1 q234e_new==1 q234f_new==1 (486 real changes made) Page # 68

70 replace RH9_treat=. if hw3!=1 (5 real changes made, 5 to missing) replace RH9_treat=0 if hw3==1 & q234a_new==. & q234b_new==. & q234c_new==. & q234d_new==. & q234e_new==. & q234f_new==. (0 real changes made) replace RH9_treat=0 if hw3==1 & RH9_treat==. (3 real changes made) label values RH9_treat ny label variable RH9_treat "Defining shock" **RH Knowledge *Generating a disaggregated RH knowledge score, based on 8 possible points gen RH_knowledge= (RH1_id + RH1_treat + RH2_treat2 + RH3_treat + RH4_treat + RH5_treat + RH6_id + RH6_treat2)/8 (1740 missing values generated) tab RH_knowledge RH_knowledg e Freq. Percent Cum Total sum RH_knowledge Variable Obs Mean Std. Dev. Min Max RH_knowledge The three variables obtained for summary provider knowledge indices are: 1. vacc_know_index (for EPI) 2. imci_index (for IMCI) 3. RH_knowledge (for RH) DISPLAY OF HEALTH WORKER KNOWLEDGE IN THE THREE AREAS (EPI, IMCI, RH) 1. EPI As we have discussed earlier that questions related to EPI (Vaccination) are asked from all the Page # 69

71 clinical staff ( Doctor, Assistant doctor, Nurse, Midwife, Assistant Midwife, Vaccinator) table province q107_new if hw1==1, c(mean vacc_know_index) Q103 Position of HW - recode Province Doctor Nurse Midwife Auxillary Midwife Assistant Doctor Vaccinat Kabul Kapisa (Output shown only for two provinces) The scores obtained above are unweighted scores for Health Worker Knowledge on EPI The weighted scores are calculated using one of the following weighting schemes: EPI No Auxilliary Midwife and No All present N Weight Assistant Doctor N Weight Doctor Doctor Nurse Nurse Midwife Midwife Auxillary Midwife Auxillary Midwife Assistant Doctor Assistant Doctor Vaccinator Vaccinator Total Total No Assistant Doctor N Weight No Auxilliary Midwife N Weight Doctor Doctor Nurse Nurse Midwife Midwife Auxillary Midwife Auxillary Midwife 0 0 Assistant Doctor 0 Assistant Doctor Vaccinator Vaccinator Total Total No Midwife N Weight Doctor Nurse Midwife 0 0 Auxillary Midwife Assistant Doctor Vaccinator Total Using the applicable weighting scheme, the unweighted scores are converted into weighted scores. The final weighted scores obtained are as follows (for the same two provinces as shown above): Province Doctor Nurse Midwife Auxillary Assistant Vaccinator TOTAL Page # 70

72 Midwife Doctor Kabul Kapisa IMCI Questions related to IMCI are assessed with three types of health workers: Doctor, Assistant doctor and Nurses. table province q107_new if hw2==1, c(mean imci_index) Q103 Position of HW - recode Province Doctor Nurse Assistant Doctor Kabul Kapisa (Output shown only for two provinces) The scores obtained above are unweighted scores for Health Worker Knowledge on IMCI The weighted scores are calculated using one of the following weighting schemes: IMCI All present N Weight No Assistant Doctor N Weight Doctor Doctor Nurse Nurse Assistant Doctor Assistant Doctor 0 0 Total Total Using the applicable weighting scheme, the unweighted scores are converted into weighted scores. The final weighted scores obtained are as follows (for the same two provinces as shown above): Province Doctor Nurse Assistant Doctor TOTAL Kabul Kapisa RH (Reproductive Health) Questions related to IMCI are assessed with two types of health workers : Doctor, Assistant doctor and Nurses table province q107_new if hw3==1, c(mean RH_knowledge) Q103 Province Position of HW - recode Midwife Auxillary Midwife Kabul Kapisa (Output shown only for two provinces) Page # 71

73 The scores obtained above are unweighted scores for Health Worker Knowledge on RH The weighted scores are calculated using the following weighting schemes RH All present N Weight Midwife Auxillary Midwife Total : Using the applicable weighting scheme, the unweighted scores are converted into weighted scores. The final weighted scores obtained are as follows (for the same two provinces as shown above): Province Midwife Auxillary Midwife TOTAL Kabul Kapisa CALCULATING THE COMPOSITE INDEX FOR HEALTH WORKER KNOWLEDGE At this stage, we have arrived at the weighted scores for the three knowledge areas in each province. In the next step, we take a simple mean of the scores in the three knowledge areas in each province to arrive at the score called Health Worker Knowledge Index. For eg. Kabul scored 85.1, 86.0 and 75.6 in EPI, IMCI and RH, respectively. Hence, Health Worker Knowledge Index of Kabul = ( ) / 3 = 82.2 Similarly, for Kapisa Health Worker Knowledge Index = ( ) / 3 = 87.4 Health Worker Knowledge Index can be calculated for all the provinces in the similar manner. Page # 72

74 12. Staff received training in last year Description This indicator measures the percentage of health worker staff who had attended in-service training within the year before the survey. Technical details Question # 120 to 154 F5 were used to calculate this indicator. Respondents who reported attending within the last 12 months any kind of in-service training were included in the calculation. A percentage of those having attended training was calculated for each health worker and weighted by the distribution of health facilities in the national sample. Average scores were calculated for each facility type in a province and were weighted by the distribution of health worker type in the national sample. These weighted scores were then added for each province to obtain the mean for that province. These weighted scores for each of the health facility were added for each province and the median was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 12 USING STATA TO PRODUCE SCORES FOR INDICATOR 12: STAFF TRAINING STEP 1: GENERATING DUPLICATE VARIABLE FOR EACH WORKER POSITION gen post =. (2233 missing values generated) FOR DOCTOR replace post = 1 if q107==1 (548 real changes made) FOR NURSE replace post=2 if q107==2 (380 real changes made) FOR MIDWIFE/ AUX. MIDWIFE replace post = 3 if (q107 == 3 q107==4) (493 real changes made) FOR ASSISTANT DOCTOR replace post = 4 if q107==5 (27 real changes made) FOR OTHERS replace post = 5 if q107==6 q107==7 (756 real changes made) STEP 2: GENERATING DUPLICATE VARIABLE FOR INDIVIDUAL ITEMS AND RECODING NO Page # 73

75 RESPONSES AS MISSING recode q120a q121a q122a q123a q124a q125a q126a q127a q128a q129a q130a q131a (2/3=0) (98/99=.) gen (rq120a rq121a rq122a rq123a rq124a rq125a rq126a rq127a rq128a rq129a rq130a rq131a) (1986 differences between q120a and rq120a) (2124 differences between q121a and rq121a) (2097 differences between q122a and rq122a) (2095 differences between q123a and rq123a) (2150 differences between q124a and rq124a) (2109 differences between q125a and rq125a) (2124 differences between q126a and rq126a) (2149 differences between q127a and rq127a) (2126 differences between q128a and rq128a) (2062 differences between q129a and rq129a) (2134 differences between q130a and rq130a) (2122 differences between q131a and rq131a) recode q132a q133a q134a q135a q136a q137a q138a q139a q140a q141a q142a q143a (2/3=0) (98/99=.) gen(rq132a rq133a rq134a rq135a rq136a rq137a rq138a rq139a rq140a rq141a rq142a rq143a) (1975 differences between q132a and rq132a) (1891 differences between q133a and rq133a) (2120 differences between q134a and rq134a) (2008 differences between q135a and rq135a) (1986 differences between q136a and rq136a) (2077 differences between q137a and rq137a) (2049 differences between q138a and rq138a) (2072 differences between q139a and rq139a) (2087 differences between q140a and rq140a) (2072 differences between q141a and rq141a) (2079 differences between q142a and rq142a) (2074 differences between q143a and rq143a) recode q144a q145a q146a q147a q148a q149a q150a q151a q152a q153a q154a (2/3=0) (98/99=.) gen(rq144a rq145a rq146a rq147a rq148a rq149a rq150a rq151a rq152a rq153a rq154a) (2095 differences between q144a and rq144a) (2075 differences between q145a and rq145a) (2101 differences between q146a and rq146a) (2132 differences between q147a and rq147a) (2142 differences between q148a and rq148a) (1939 differences between q149a and rq149a) (2139 differences between q150a and rq150a) (2149 differences between q151a and rq151a) (1900 differences between q152a and rq152a) (2119 differences between q153a and rq153a) (2126 differences between q154a and rq154a) STEP 3: GENERATING NEW VARIABLE FOR TRAINING (i12) gen i12 = 0 REPLACING TRAINING = 1 IF STAFF HAS RECEIVED TRAINING IN LESS THAN 1 YEAR Page # 74

76 replace i12=1 if (rq120a==1 rq121a==1 rq122a==1 rq123a==1 rq124a==1 rq125a==1 rq126a==1 rq127a==1 rq128a==1 rq129a==1 rq130a==1 rq131a==1 rq132a==1 rq133a==1 rq134a==1 rq135a==1 rq136a==1 rq137a==1 rq138a==1 rq139a==1 rq140a==1 rq141a==1 rq142a==1 rq143a==1 rq144a==1 rq145a==1 rq146a==1 rq147a==1 rq148a==1 rq149a==1 rq150a==1 rq151a==1 rq152a==1 rq153a==1 rq154a==1) (1139 real changes made) REPLACING TRAIING AS MISSING IF ANY OF THE INDIVIDUAL RESPONSES ARE MISSING replace i12=. if (rq120a==. & rq121a==. & rq122a==. & rq123a==. & rq124a==. & rq125a==. & rq126a==. & rq127a==. & rq128a==. & rq129a==. & rq130a==. & rq131a==. & rq132a==. & rq133a==. & rq134a==. & rq135a==. & rq136a==. & rq137a==. & rq138a==. & rq139a==. & rq140a==. & rq141a==. & rq142a==. & rq143a==. & rq144a==. & rq145a==. & rq146a==. & rq147a==. & rq148a==. & rq149a==. & rq150a==. & rq151a==. & rq152a==. & rq153a==. & rq154a==.) (1 real change made, 1 to missing) STEP 4: LABELING OF VARIABLES label var training "Staff received training in last one year" STEP 5: DISPLAY OF INDICATOR STAFF RECEIVED TRAINING IN LESS THAN ONE YEAR table province post if post!=5, c(mean i12) Un-weighted Scores Province Doctor Nurse Midwife/ Aux. Assn. Doc. Takhar Others MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN IN TABLE 1.2b Weighted Scores Province Doctor Nurse Midwife/ Aux. Assn. Doc. Takhar Others ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province score Province Score Takhar 45.6 Page # 75

77 13. HMIS Use Index Description This indicator measures the availability and use of the Health Management Information System by looking at the following 5 items. 1. Monthly integrated activity report (MIAR) 2. Facility status report (FSR) 3. Notifiable diseases report Technical Details Questions # 409, 412 & 413 from F7 were used to calculate this indicator. If MIAR and FSR was present and complete for the last month, at the time of teams visit to the facility, a health facility received 1 point. For Notifiable Disease Report, if the facility reported only for the presence of the report at the facility, the facility received 1 point. Scores on each of these 3 items were combined to create an index with scores ranging from 0-3. The scores were converted into percentages by dividing the scores by 3. Average scores were calculated for each facility type within a province and weighted according to the national sample health facility distribution. These weighted scores were then added for each province to obtain the mean for that province. These weighted scores for each of the health facility were added for each province and the mean was calculated for the country. The provincial scores are then put in ascending order of magnitude and lower and upper quintiles are marked in red and green respectively. STATA DEMONSTRATION 13 USING STATA TO PRODUCE SCORES FOR INDICATOR 13: HMIS USE INDEX STEP 1: RECODING MIAR AND FSR IF PRESENT AND COMPLETE 1 ELSE 0, NO RESPONSES/DONOT KNOW AS MISSING AND NDR IF PRESENT 1 ELSE 0 codebook q409 q412 q413 recode q409 q412 (2/4=0) (98/99 =.) (q409: 35 changes made) (q412: 72 changes made) recode q413 (1/2=1) (3/4=0) (98/99=.) (q413: 166 changes made) codebook q409 q412 q413 STEP 2: GENERATE NEW VARIABLE i13 FOR HMIS INDEX BY DIVIDING 3 TO CONVERT IT TO A PERCENTAGE gen i13 = (q409 + q412 + q413)/3 (2 missing values generated) Page # 76

78 STEP 3: TABULATING HMIS INDEX VARIABLE BY PROVINCE AND FACILITY TYPE table province fac_type, c(mean i13) Un-weighted Scores BHC CHC DH Baghlan MULTIPLY UN-WEIGHTED SCORES WITH WEIGHTS GIVEN ON TABLE 1.1a Weighted Scores BHC CHC DH Baghlan ADD SCORES BY FACILITY TYPE AND MULTIPLY BY 100 TO GET PROVINCIAL SCORES Province Score Province Score Baghlan Clinical Guidelines Index Description This indicator measures the availability of Clinical Guidelines at various facilities under the following 9 items. Technical Details 1. Integrated Management of Childhood Illness (IMCI) 2. Growth monitoring 3. Tuberculosis diagnosis and treatment 4. Malaria 5. ORT corner 6. Patient education materials 7. HMIS guidelines 8. Immunization schedule 9. Family planning guidelines Questions # 900, and 519 from F7 were used for this indicator. Scores on each of these 9 items were combined to create a simple index with scores ranging from 0-9. Average scores were calculated for each facility type within a province and weighted according to the national sample health facility distribution. These weighted scores were then added for each province to obtain the mean for that province. Page # 77