Performance Assessment of Health Workers Training in Routine Immunization in India

Transcription

1 1

2 Performance Assessment of Health Workers Training in Routine Immunization in India (WHO and NIHFW collaborative study) Study Report December,

3 Study Report December-2009 Performance Assessment of Health Workers Training in Routine Immunization in India (WHO and NIHFW collaborative study) Principal Investigators Chief Investigators Co Investigators Dr. Deoki Nandan, Director, NIHFW Dr. Hamid Jafari, Project Manager, WHO-NPSP Dr. Utsuk Datta, Professor, Education & Training, NIHFW Dr. Sunil Bahl, Dy. Project Manager, WHO-NPSP Dr. Renu Paruthi, Training focal Person, WHO NPSP Prof. M. Bhattacharya, Professor, Dept. of CHA, NIHFW Dr. Sanjay Gupta, Associate Professor, Dept. of CHA, NIHFW Dr. Balwinder Singh, Ag. National RI team leader, WHO NPSP Dr. P.K. Roy, Monitoring & Evaluation, Focal Person, WHO NPSP Dr. P. Deepak, Consultant Immunization training, NIHFW 3

4 Table of Contents Topics Page No. Foreword 4 Abbreviations 5 Executive Summary Introduction 6 11 Objectives of the Study 12 Methodology 12 Study Findings 14 Major Conclusions and Recommendations 37 Annex 1: State wise summary of observations and suggestions by the study teams 41 Annex 2: Guidelines for Immunization Training of Health Workers 53 Annex 3: The Program schedule for Immunization Training of Health Workers 53 Annex 4: Day wise tasks for each study team for data collection 54 Annex 5: Study tools numbers 1 6 including the instructions to fill 56 Annex 6: List of Study Team Members 69 4

5 5

6 Abbreviations ADS AEFI ANM ANMTC ASHA AVD AWW CHC DIO GoI HWs HA (M) HA (F) HQ IPC LHV MoHFW MP NRHM NIHFW NPSP PATH PHC PNA PRI SEARO SHG UIP UNICEF UP WHO Auto Disable Syringes Adverse Events Following Immunization Auxiliary Nurse Midwife Auxiliary Nurse Midwife Training Centre Accredited Social Health Activist Alternate Vaccine Delivery Anganwadi workers Community Health Centre District Immunization Officer Government of India Health Workers Health Assistant (Male) Health Assistant (Female) Head Quarters Inter-personal Communication Lady Health Visitor Ministry of Health and Family Welfare Madhya Pradesh National Rural Health Mission National Institute of Health and Family Welfare National Polio Surveillance Project Program for Appropriate Technology in Health Primary Health Centre Performance Needs Assessment Panchayati Raj Institution South East Asia Regional Organization Self Help Group Universal Immunization Programme United Nations Children's Fund Uttar Pradesh World Health Organization 6

7 Executive Summary Baseline study on Performance Needs Assessment (PNA) of Health Workers was conducted in late 2005, across 8 states and 40 districts to identify the areas for immunization training. The results of this survey lead to the development of the Immunization Handbook for Health Workers and the related Facilitators Guide, published by MoHFW in August, An initial period of state-level training of trainers (ToTs) was followed by health worker training in districts and by December 2008, approximately 100,000 of the total 200,000 (50%) HWs in India were trained in the country. The overall objective of this study was to assess the level of health workers performance in providing immunization services following the introduction of the Immunization Handbook for Health Workers and the Facilitator s Guide (GoI, 2006). The study was conducted jointly by WHO-NPSP and NIHFW. Data collection was done in two phases during June (8-13) and July (13-20), The specific objectives of the study were as follows: 1) To assess the performance (level of knowledge and skills) of health workers after training in immunization. 2) To seek the opinion of trainers and the trained health workers on course curriculum and methodology followed during the training. 3) To seek the opinion of supervisors and beneficiaries about the performance and job behavior of health workers. 4) To find out the non-training issues that would enhance on the job performance with respect to immunization service provision. 5) To suggest interventions (if any) for improving the performance of health workers (including modifications in future training courses) The study covered a sample of trained health workers (ANMs, LHVs, Male health workers and health assistants), PHC medical officers, District Immunization officers / Training Coordinators and beneficiaries/care givers from the seven states, Bihar, Jharkhand, Madhya Pradesh, Orissa, Rajasthan, UP and Uttarakhand. These same states had conducted the PNA study. Hence, where appropriate comparing the results of the two surveys has been done, however, it must be noted that because of the different sample sizes and methodologies used, comparisons are illustrative and not statistically comparable. Forty study teams, one for each district were identified amongst officers from NIHFW, WHO- NPSP, WHO-SEARO, WHO-HQ, UNICEF, IMMbasics, Medical Colleges, Training Centers and State governments. Data was collected from total of 2292 respondents including 40 District Trainers, 82 Block Medical officers, 313 health workers and 1857 beneficiaries/caregivers through interviews, observation of health workers conducting immunization sessions and record study. 7

8 Major conclusions drawn from the study and the recommendations are as follows: 1. Training All the states with the exception of Jharkhand have trained more than 80% of female health workers (ANMs) on vaccination of children and pregnant women, while the newly recruited contractual female health workers and other categories as male health workers, health assistants, cold chain handlers, data handlers etc are still left to be trained for their specific tasks. State governments need to release funds to the districts to continue the in-service training for all health workers including HW (male), HA (male), HA (female) cold chain and data handlers in their specific tasks at PHC and District level as well as newly recruited contractual health workers. Sixty-five percent of trainers deemed the course contents were adequate and rationally developed. Suggested topics to be included in the Handbook were details on VPDs, communication techniques and details on newer vaccines. One in four of those interviewed felt that more field based sessions with practical demonstration of preparing due list of beneficiaries and VHNDs should be included. Training material needs to be updated based on recent policy changes and more emphasis is required on improving the communication skills (in the handbook as well as in the facilitators guide). Eighty percent of the districts had training centers and class rooms available, however, only half had hostel rooms and transport facilities. Residential arrangements were made by 35% of the districts. There is need to establish or strengthen the logistical capacity and infrastructure of training facilities and residential arrangements need to be made in each district for training. The average batch size for HW training was 26, well above the norm of 15. During field visits, only 38% of the health workers reported having the opportunity to practice injections. While all training centers distributed Immunization Handbooks in local language, only 64% of HWs reported viewing the Immunization film during training. States need to enforce guidelines for a batch size of 15 and at least 3 trainers for each training course. Hands-on opportunities must be provided to the trainees by taking the participants in small batches to 3-4 different session sites. Greater oversight of training sessions by state and district officials as well as partners is necessary. Sixty percent of health workers reported that only 2 to 3 trainers were involved on both days of training. Trainers skills were found to be weak in conditioning of ice-packs, use of hubcutters, AEFI management and tracking of drop outs. Trainers from Orissa and Rajasthan performed better as compared to the other states on all parameters. States need to train more trainers to ensure adequate number of trainers in all the districts. Trainers skills need to be evaluated and strengthened through annual refresher trainings. One district level officer should be designated as nodal officer for coordinating the immunization training of all health functionaries. 8

9 2. Performance of health workers after training in Immunization Injection administration skills of Health Workers appear to have improved since the PNA study. 85% of HWs were Injecting vaccine using the correct route per antigen; 75% were allowing dose to self disperse instead of massaging and 67% were maintaining aseptic technique. 98% of the health workers were found competent in using ADS and 96% in using new disposable syringe for each reconstitution. However, only half of observed health workers cut each syringe with hubcutters immediately after use. Regular training and retraining of health workers in RI is required every year. There is a need to provide hands on training to health workers to practice injection administration techniques. Documentation (recording and reporting) skills of health workers also improved after training as compared to the baseline PNA study with 73% of all health workers documenting each vaccination correctly and completely. However, reporting of VPDs and AEFIs was found to be very poor across all the states. HWs need to be supervised to ensure that they prepare the due lists, fill counterfoils and update the Immunization registers regularly. HWs should be sensitized and encouraged for reporting of VPDs and AEFIs. Interpersonal Communication skills of health workers were found to be weak even after training. 44% were welcoming beneficiaries; 40% were explaining potential adverse events following immunization and 39% were discussing with beneficiaries/parents about the next visit. Only 18% were explaining what vaccines would be given and the VPDs prevented; 13% were screening the beneficiaries for contraindications and 15% were asking the beneficiaries to wait for minutes after vaccination. Need to improve communication skills and technical knowledge of the health workers by ensuring supportive supervision and on the job training by MO/PHC during immunization sessions. As a constant reminder for health functionaries, key IPC messages in local language should be displayed at the session site. While 93% of beneficiaries/caregivers knew about the place of immunization session, only 49% caregivers knew when to go for next due vaccine. Three-fourths of care givers were reminded for vaccination prior to vaccination day by the social mobilizer and 70% children had received age appropriate vaccines. IEC and IPC in the community need to be improved. HWs need to share the due list of beneficiaries with ASHA/AWW for tracking of drop-outs. ASHAs need to be trained for better community mobilization especially in the resistant group. 3. Immunization Program support to HWs Coverage / Microplanning: Eighty six percent of the sessions were being held as per the RI microplans; however, a map of the catchment area was available in only 39% of the PHCs. Estimation of number of beneficiaries and logistics as a part of microplan was limited (61% of 9

10 PHCs). Though roster of HWs was available at 88% of the PHCs, the Alternate Vaccine Delivery plan was available in only 61% of the PHCs. Proper Microplans with maps are required so that no area is missed. Maps showing the subcentres, distance of session sites from ILR points and alternate vaccine delivery plan need to be displayed at each PHC. Cold chain and logistics management: Designated and trained cold chain handlers were not available at the PHCs. Vaccines were delivered through Alternate Vaccine Delivery System to 71 % of session sites. 73% of PHCs were correctly maintaining the stock registers; 64% were correctly maintaining the temperature log books and only 48% were conditioning the icepacks correctly. Stock-outs or shortage of vaccines or syringes in last 3 months were reported by 37% of PHCs covering all the states. Twice daily recording of temperature in the log books needs to be monitored. Urgently designate and train the cold chain handlers in their specific tasks. Nonfunctioning cold chain equipment needs to be repaired or replaced. Ensure regular supplies of immunization cards, registers, tracking bags and coverage monitoring charts. Injection safety and waste disposal: Forty nine percent HWs were cutting AD syringes immediately after use and 35% were using red bags to keep cut AD syringes. Availability of functional hub cutters at PHC (54%) and Session site (53%) was poor. Availability of red bags (36%) and black bags (32%) was also poor at session site. Though disposal pits were made by 54% of PHCs, disinfection was practiced only by 30% PHCs; syringes and needles were thrown into these pits meant for sharps only; burning and burying of the immunization waste was also practiced. Only 68% of the trainers had good knowledge of the use of hub cutters and safe disposal of immunization waste. States need to establish a system of collection of segregated waste from session site to PHC for disinfection and proper disposal; ensure regular supplies of hubcutters, bleaching solution, red and black bags and also construction and proper utilization of waste pits for disposal of sharps. Monitoring and supervision by medical officers needs to be ensured. AEFIs: Only 53% of the trainers of health workers had good knowledge on the management of AEFIs and 69% of the health workers knew proper treatment of minor AEFIs. However, only 40% of health workers explained potential AEFIs to care givers. Only 7% of the PHCs had reported AEFIs in the MPRs of last three months preceding the study. Training surrounding all aspects of AEFI needs to be strengthened to ensure that medical officers regularly sensitize the health workers on reporting and management of AEFIs and that HWs explain potential AEFIs and their management consistently to each beneficiary / caregiver. Tracking and mobilization of beneficiaries: During observed sessions, only 39% of health workers discussed with beneficiaries /parents the date of the next visit. Approximately 30% of beneficiaries did not receive age-appropriate vaccinations. Only 4% PHCs displayed coverage monitoring charts and only 50% of trainers had adequate knowledge about tracking of dropouts. 10

11 Preparing due lists, updating counterfoils and using tracking bags should be standardized for tracking beneficiaries. HWs need to update immunization register by including information from the records of AWW/ASHA as well as new born tracking booklets of SIA. Coverage monitoring charts should be displayed in every PHC and sub center and this should be ensured through supportive supervision and on the job training by MO/PHC during immunization sessions. Monitoring and Supervision: Though medical officers of the PHC reported monitoring on an average 4 session sites per month and other supervisors monitored around 7 session sites per month during last 3 months, no records to support the supervisory visits were available in 45% of the PHCs. Fifty six percent of health workers reported that MO had visited their session sites in the last 3 months while 63% reported the visit of other supervisors. Provide mobility support to medical officers for the field visit and supervision. Train all supervisors on monitoring and supervision of all the health functionaries involved in immunization services. The monitoring by BLOCK TEAM and by DISTRICT TEAM should be made mandatory; this needs specific guidelines and tools to be prepared and shared with all states and districts. Way Forward: Update training materials at national level for training the health workers, cold chain and data handlers in the states. Use innovative training methodologies e.g. developing web based training packages for e- learning, videos etc. States need to provide refresher training to the trainers as well as all health functionaries every year and orientation/induction training for the new staff. Annual State PIPs under NRHM need to include all the immunization training related activities. Regular monitoring to understand weaknesses and tailored supportive supervision of all health functionaries is required to reinforce training and improve quality of services and ultimately immunization coverage rates. 11

12 1. Introduction Immunization Training for Health Workers (HWs) was identified as a key recommendation by both the Universal Immunization Programme (UIP) Review (2004) and the Ministry of Health and Family Welfare s (MoHFW) Multi Year Plan for immunization ( ). Following this, Government of India (GoI) formed a National Core Committee for Immunization training in Committee members, including representatives from MoHFW, NIHFW, WHO, UNICEF, PATH, IMMbasics and CARE, reviewed existing immunization training materials and conducted a Performance Needs Assessment (PNA) of Health Workers, across 8 states and 40 districts, in late The following areas were identified for immunization training: Micro-planning Vaccine administration techniques Cold Chain maintenance Recording and reporting Use of AD syringes Safe disposal of used syringes and needles Management of AEFIs IPC and Counseling etc. All these areas were included in the Immunization Handbook for Health Workers and the related Facilitators Guide which were developed after extensive inputs from stakeholders, field testing and a national consultative workshop, and published by MoHFW in August, Translation and printing in local languages was done by state governments with help from partners. States prepared their training plans and calendars and MoHFW provided funds to states (under NRHM) for the training. For the HWs training, the cascade model of training was adopted, with members from the National Core Committee conducting Training of Trainers (ToTs) courses at the state level, followed by the two-day actual district-level training of front line HWs. Emphasis was on conducting participatory training, with extensive use of hands-on practice and field visits. ToTs were held mainly during 2007, training around 4600 trainers in 33 states. Majority of the states started Health workers training by the end of By December, 2008, around 100,000 of the total 200,000 (50%) HWs were trained in the country. It was considered important to understand whether the training of health workers was able to close the gaps identified by PNA and, how well the trainees were able to perform their job responsibilities of providing immunization services to the required standards. In this regard, WHO/NPSP proposed to conduct a study in collaboration with NIHFW to assess the performance of trained health workers in the same seven states where the PNA study was conducted. Government of India accorded its approval for conducting an independent study. 12

13 Study limitations: There are several important limitations of this study which must be mentioned. First, the sampling methodology employed and size of the sample selected for each study component does not allow generalization of results to larger sub-populations within individual states. Likewise, it is not possible to generalize study findings from the seven study states to the rest of the country. Moreover, without a concurrent comparison state, i.e., a state where training with the Handbook had not yet taken place, it is impossible to determine whether health worker performance is attributed to training or other programmatic or non-programmatic factors. Second, the different sampling methodologies employed by the two studies obviate direct statistical comparison of results. Thus, a true baseline does not exist and a comparison of results between the PNA and the HW assessment are illustrative only. 2. Objectives General Objective: To study the level of task performance of health workers in providing immunization services after training with Immunization Handbook for Health Workers and the Facilitator s Guide (GoI, 2006) Specific Objectives: 1) To assess the performance (level of knowledge and skills) of health workers after training in immunization. 2) To seek the opinion of trainers and the trained health workers on course curriculum and methodology followed during the training. 3) To seek the opinion of supervisors and beneficiaries about the performance and job behavior of health workers. 4) To find out the non-training issues that would enhance on the job performance with respect to immunization service provision. 5) To suggest interventions (if any) for improving the performance of health workers (including modifications in future training courses) 3. Methodology Study Area: PNA Study was undertaken in 8 states of the country. The current study proposed to include seven states as of PNA study i.e. UP, MP, Bihar, Jharkhand, Uttarakhand, Rajasthan and Orissa, where 50% to 90% training of Health Workers had been completed. Andhra Pradesh was not included in the study because the health workers training had not started in that state. 13

14 Study Population: It comprised of trained health workers (ANMs, LHVs, Male health workers and health assistants); PHC medical officer; District Immunization officer/training Coordinator and beneficiaries / care givers. Sampling Technique and Sample size: 40 districts were selected from 7 states after listing the district-wise performance of health workers training for each state. Then colored maps showing the districts with training performance of <50%; 50 80%; 80 90% and >90% were prepared. This was followed by random selection of approximately one sixth of districts in each state from amongst the districts with more than 80% training performance, covering all geographical areas. In each identified district, two blocks were selected randomly, one near the district HQ (within 15 kms.) and other distant from the district HQ (more than 15 kms). From each block, four health workers were selected randomly. Thus, 8 Health Workers were selected (who had already been trained) from each district. For the selection of health workers, the block was divided in to 4 zones and one trained health worker was selected from each zone randomly. The PHC Head Quarter was to be excluded in the process. On the immunization session day, each team member visited two health workers for interview and observation during the immunization session being conducted. Then each team member randomly visited four households with children of 0-2 years of age-group in each health worker s area to interview the beneficiaries / caregivers. Sample size for the study was decided as 40 district training coordinators / DIOs, 80 block medical officers, 320 health workers and 1920 beneficiaries/care givers. Data collection schedule: The data was collected in two phases during June 8-13 and July 13-20, 2009 State (number of Districts Phase-1 (14 districts from 3 states) districts) UP(6) Ambedkarnagar, Basti, Mahoba, Fatehpur, Jalaun, Jaunpur Bihar(6) Buxar, Champaran East, Darbhanga, Gaya, Khagaria, Kishanganj Uttarakhand (2) Champawat, Tehri Garhwal Phase-2 (26 districts from 5 states) UP (6) Jharkhand (5) Rajasthan (5) MP(6) Orissa(4) Rampur, Moradabad, Lucknow, Saharanpur, Badaun, Sonbhadra Bokaro, Gumla, Hazaribagh, Godda, Palamu Jaisalmer, Dholpur, Pali, Bikaner, Bhilwara Vidisha, Bhopal, Anuppur, Betul, Indore, Ratlam Balasore, Gajapati, Nayagarh, Nuapada 14

15 Data collection Tools: The following tools were finalized after field testing by 2 member teams in 4 districts of three states (UP, Bihar and Jharkhand) followed by one day finalization workshop on : 1 Study Tool-1 Interview schedule for District Immunization Officer/ District Training Coordinator/Trainer 2 Study Tool-2 Interview schedule for Block /PHC Medical officer 3 Study Tool-3 Checklist for Record Study (Look for Monthly Progress Reports of the block for last 3 months) 4 Study Tool-4 Interview Schedule for Health Worker who received Immunization training 5 Study Tool-5 Checklist for observing the skills of Health Worker in conducting immunization session 6 Study Tool-6 Checklist for House to house visit to assess knowledge of care givers Data collection work: 40 study teams, one for each district were identified amongst officers from NIHFW, WHO-NPSP, WHO-SEARO, WHO-HQ, UNICEF, IMMbasics, Medical Colleges, Training Centers and State governments. Each team comprised of 2 (4 for Orissa) members. Each team covered one district in 3-5 working days. A detailed scheme for data collection and the day wise tasks of study teams are given at Annex-4. Before the data collection, one day orientation training was organized at NIHFW on 8 th June and 10 th July, 09 for all the team members. One day debriefing meeting was held after data collection on 15 th June and 22 nd July, 09 at NIHFW for one member from each team to submit the filled in data collection tools, the summary of observations and the financial expenditures. Data Validation and Data Analysis: Data validation exercise was conducted for all the data collected. This included crosschecking and matching the data from hard copies in to the soft copies and clarifications sought after interacting with the investigators. Then, state wise data was compiled and analyzed based on each objective of the study, leading to preparation of graphs and tables for inclusion into the report. 4. Study Findings The observations of the study are presented under the following heads- A. General Profile (Study Tools-1 and 4) B. Performance (level of knowledge and skills) of health workers after training in immunization. (Study Tools-4 and 5) C. The opinion of trainers and the trained health workers on course curriculum and methodology followed during the training. (Study Tools-1 and 4) D. The opinion of supervisors and beneficiaries about the performance and job behavior of health workers. (Study Tools-2 and 6) E. Non-training issues that would enhance on the job performance with respect to immunization service provision. (Study Tool-2) 15

16 A. General Profile: 1. Data Collected: Data was collected from 2292 respondents through interviews, observation of health workers conducting immunization sessions and record study. (Table 1) Table 1: Number of respondents of the study States District Block Health Exit Trainers MOs Workers Interviews Beneficiaries Bihar Jharkhand MP Orissa Rajasthan UP Uttarakhand India Total years in service of Health Worker: Majority of Health Workers had more than 10 yrs of job experience as shown in Figure 1. Figure 1: Total years in service of Health Worker (n=313) <2yrs, 4.2% 2-5 yrs, 8.9% 3. Number of health workers posted at the sub-center: 48% of the sub centers had around 2 or more HWs posted. Around 2/3 rd of the sub >10yrs, 71.2% 5-10 yrs, 15.7% centers in MP, Orissa and Jharkhand and 49% in Bihar had 2 or more HWs. In UP, Rajasthan and Uttarakhand, majority of sub centers had only one HW. 16

17 4. Average population covered by a Health Worker posted at the sub-center: In each of the states, HWs were covering population much more than the national population norms for a sub center (5000 in general and 3000 in hilly, tribal and backward areas) as shown in Figure 2. Figure 2: Average population covered by a Health Worker India (n-313) Bihar (n=48) Jharkhand (n=40) M P (n=44) Orissa (n=32) Rajasthan (n=39) U.P. (n-96) Uttrakhand (n=14) 5. Health Workers staying at the Sub-center: Overall 32% of HWs were staying at the subcenters while in Orissa, 75% HWs stayed at the subcentre. It is shown in Figure 3. Figure 3 : Health Workers staying at the Sub-center Percent India (n=313) Bihar (n=48) Jharkhand (n=40) M P (n=44) Orissa (n=32) Rajasthan (n=39) U.P. (n-96) Uttrakhand (n=14) 17

18 6. Any other immunization training received in last three years: 72% of the health workers had not received any other immunization training in last three years. 7. Percentage of Health Workers trained: All the states had trained majority of HWs (F) and also HAs (F) though the percentage of trained HA (F) was low in MP, UP and Jharkhand. HWs (M) were not trained by UP and Jharkhand. HAs (M) were trained only by Orissa and Uttarakhand. (Table 2) Table 2: Training status of Health Workers at district level (state wise % trained) Category of Health Workers Bihar Jhark hand MP Orissa Rajast han UP UA India ANM (HW-F) LHV (HA-F) HW-M HA-M Any other category Total

19 B. Performance (level of knowledge and skills) of health workers after training in immunization: Each study team member observed the Health Workers at the session site providing immunization services to at least two beneficiaries. Then, the assessment of the health worker s performance was done on each parameter, whether she was competent or needed to improve. The health workers were also asked questions to assess their knowledge in immunization. 1. Skills of trained Health Workers in providing immunization services as Compared to PNA study Injection Administration Skills: Significant Improvement was observed in maintaining aseptic technique, using the correct route for vaccination and Percent Figure 4 : Injection Administration Skills of trained Health Workers Maintains aseptic technique throughout Injects vaccine using the correct route for the vaccine Allows dose to selfdisperse instead of massaging PNA-2005 (n=200) HW Trg Evaluation-2009 (n=306) Figure 5 : Documentation Skills of trained Health Workers Percent Verifies beneficieries records for vaccination Checks that it is the correct date for the vaccination Documents each vaccination correctly and completely PNA-2005 (n=200) HW Trg Evaluation-2009 (n=306) 19

20 allowing the dose to self disperse instead of massaging. (Figure 4) Documentation Skills: Improvement was observed in verifying the beneficiaries records for vaccination, checking for correct date and documenting each vaccine correctly and completely. (Figure 5) Interpersonal Communication Skills: It was still a weak area for the Health Workers. Though marginal improvement was observed in welcoming beneficiaries, explaining potential AEFIs and discussing about the next visit, deterioration was seen in explaining about the vaccines to be given and VPDs prevented as well as in screening for contraindications.(figure 6) State wise comparisons are given in Table-3 Percent Figure 6 : Interpersonal Communication Skills of trained Health Workers Welcomes beneficieries Explains what vaccine(s) will be given Screens for contraindications Explains potential adverse events following immunization PNA-2005 (n=200) HW Trg Evaluation-2009 (n=306) HW discusses with beneficieries /parents about next visit 20

21 Table 3: Skills of HWs in providing immunization services as Compared to PNA study (State wise % of HWs) Bihar Jharkhand MP Orissa Rajasthan UP Uttarakhand India Skills Indicators PNA 2005 (n=47) 2009 study (n=48) PNA 2005 (n=20) 2009 study (n=40) PNA 2005 (n=14) 2009 study (n=42) PNA 2005 (n=15) 2009 study (n=32) PNA 2005 (n=30) 2009 study (n=37) PNA 2005 (n=70) 2009 study (n=95) PNA 2005 (n=4) 2009 study (n=12) PNA 2005 (n=200) 2009 study (n=306) Injection Administration Documentation Maintains aseptic technique throughout Injects vaccine using the correct route for vaccine(im/sc/id) Allows dose to selfdisperse instead of massaging Verifies beneficiaries records for vaccination Checks that it is the correct date for vaccination Documents each immunization correctly and completely Welcome Beneficiaries Interpersonal Communication Explains what vaccines(s) will be given Screens for contraindications Explains potential adverse events following immunization HW discusses with beneficiaries/parents about next visit

22 2. Skills of trained Health Workers in providing immunization services in areas in addition to PNA study Skills in safe injections and waste disposal: Almost all the health workers were competent in using AD Syringes and new disposable syringe for each reconstitution. Major improvement is required in use of hub cutters as well as red and black bags for waste disposal. (Figure 7) Figure 7 : Skills of Health Workers in providing Immunization services: Observations at session site (n=306) Percent Uses AD syringe to give vaccination Uses new disposable syringe for each reconstitution of the vaccines 48.8 Cuts each AD and Disposable syringe with hub cutter immediately after use 35 Used syringes are kept in red bag for sending back to PHC Post vaccination activities: 56.1% of health workers were updating the counterfoils after vaccination while only 14.7% were asking the beneficiaries to wait for minutes after vaccination. (Figure 8) Figure 8 : Skills of Health Workers in providing Immunization services: Observations at session site (n=306) Percent Updates counterfoil of the beneficiary after vaccination Asks the beneficiaries to wait for minutes after vaccination Immunization schedule was followed correctly as checked from 2 beneficiaries cards 22

23 3. Exit interview of the care givers to check the immunization status from the immunization cards As shown in Figure 8, 70.4% of the Health Workers were found competent in providing vaccines according to the immunization schedule and recording them in the immunization card. State wise comparisons are given in Table 4 below. Table 4: Skills of HWs in providing immunization services in areas in addition to PNA study (State wise % of HWs) Indicators Uses AD Syringe to give vaccination Bihar (n=48) Jhark hand (n=40) MP (n=42) Orissa (n=32) Rajast han (n=37) UP (n=95) UA (n=12) India (n=30 6) Uses new disposable syringe for each reconstitution. Cuts each AD and Disposable syringe with hub cutter immediately after use Used syringes (after cutting the needle) are kept in red bag for sending back to PHC Updates the counterfoils after vaccination Asks the beneficiaries to wait for minutes after vaccination Providing vaccines according to the immunization schedule and recording them in the immunization card Knowledge of the trained health workers in immunization During interview, health workers were asked open ended questions; their responses were noted and judged as Figure 9 : Knowledge of the trained Health Workers (n=313) Percent correct responses Vaccination of a 16 months old child Vaccination of a child coming after a gap of 6 months Dose of Vit A M ethods of Route & site of Tracking dropout BCG children Route & site of DPT Route & site of M easles 23

24 correct if they matched with the correct answers. Their knowledge regarding tracking of drop outs and the vaccination schedule of children coming late for vaccination was found to be poor. (Figure 9). Table 5 gives the state wise comparisons. Table 5: Knowledge of the trained health workers in immunization (State wise % of HWs with correct responses) Questions If a child comes for vaccination for the first time at 16 months of age, what should be given? A child received BCG, DPT1 and OPV1 at the age of 1 and half months and then comes again after a gap of 6 mths. Which vaccines will you give? Dose of Vit A solution for a child above 1yr of age? How can you track drop out children What will you do if a child comes with mild fever, pain and swelling at the site of injection What is the route and site for administration of BCG What is the route and site for administration of DPT What is the route and site for administration of Measles Bihar (n=48) Jhark hand (n=40) MP (n=44) Orissa (n=32) Rajast han (n=39) UP (n=96) UA (n=14) India (n=31 3) Reporting of AEFIs and VPDs Monthly progress reports of last 3 months from the health workers were checked at the PHC to see if they had reported any AEFI or VPD in the last 3 months. There was either no reporting or very poor reporting of these across all the states. (Table 6) Table 6: Reporting of VPDs and AEFIs from PHCs (State wise % of PHCs) Indicators More than 80% of planned sessions held Any AEFI reported in last 3 calendar months Any VPD reported in last 3 calendar months Bihar (n=12) Jhark hand (n=10) MP (n=12) Orissa (n=8) Rajast han (n=10) UP (n=24) UA (n=6) India (n=82)

25 6. Performance of trained health workers at the session site 86% of the sessions were being held as per the RI microplans. Due list of beneficiaries was available with 60% of health workers. However, less than half of HWs in UP, Bihar and Uttarakhand had the due lists available with them. 76% of Health Workers had kept the reconstituted vials in shade on the ice packs and 79% wrote the time of reconstitution on BCG and Measles vials. 7. Examples of improvement in immunization practices after training as informed by HWs (Figure 10) 70% of the HWs reported improvement in their injection techniques and site of injection after the training. Only 4% reported improvement in their communication and social mobilization activities; same number reported improved knowledge in AEFIs. Figure 10 : Improvement in the immunization practices of HWs after training (n=313) New injection techniques and sites 70 Safe disposal of injection waste Cold chain maintenance in session Tracking, data interpretation, recording and reporting Safe injections Immunization schedule and vaccine dosage M icroplanning and conduction of sessions Communication and mobilization Contraindications/AEFIs Any new initiatives/activities conducted to improve community involvement after training (Figure 11) 43% HWs did not respond. 26% reported better coordination with ASHAs, AWWs and other volunteers. 1/4 th informed holding monthly meetings with different stakeholders (community, parents, members of SHGs, PRI representatives, etc) while only 7% HWs reported using inter personal communication to convince parents. 25

26 Figure 11 : New initiatives/activities conducted to improve community involvement after Training (n=313) No response 43.1 Better coordination with ASHAs/AWWs/other volunteers 26.2 M onthly meetings held with different stakeholders 23.6 Others 8.6 Influencing parents through better IPC 7 26

27 C. The opinion of trainers and the trained health workers on course curriculum and methodology followed during the training District Training coordinator/dio/trainer were interviewed and the records and facilities were observed to assess the quality of training provided to the health workers. Health workers were also interviewed to find out their opinion on the quality of training and their suggestions for improving future training. 1. Opinion of trainers about the quality of Training Courses conducted: Average batch size for HW training was found to be 26, much more than the norm of 15 in each of the states (Figure 12) Last training batch was conducted 1-2 years back as reported by 57.5% of trainers. However, 37.5% reported last training batch within one year. Figure 12: Average number of Trainees per batch India (n=40) Bihar (6) Jharkhand (5) MP (6) Orissa (4) Rajasthan (n=5) UP (n=12) Uttrakhand (n=2) Average number of TOT trained trainers available in the districts for HW training was 5 but only 3 in Bihar and Uttarakhand. (Figure 13) 27

28 Average number of trainers involved in HW training on both days was only 2 to Figure 13 : Number of TOT trained trainers available in the districts India (n=40) Bihar (6) Jharkhand (5) MP (6) Orissa (4) Rajasthan (n=5) UP (n=12) Uttrakhand (n=2) 3 in 64% of the districts as informed by the trainers. (Figure 14) Figure 14 : Involvement of Trainers on both days of Training Percent India (n=40) Bihar (n=6) Jharkhand (n=5) MP (n=6) Orissa(n=4) Rajasthan (n=5) UP (n=12) Uttarakhand (n=2) 2 to 3 Trainers 4 or More Trainers Training was residential only in 35% of the districts. Orissa had made residential arrangements in all the districts; UP and Rajasthan in around 40% of the districts while Bihar in one third. No residential arrangements were made by Jharkhand, Uttarakhand and 5 out of 6 districts in MP. All the Trainers except in Jharkhand and Uttarakhand reported that they had organized field visits during training. 28

29 All the training centers distributed Immunization Handbooks in local language, 70% distributed handouts from the facilitators guide and 80% gave certificates to the participants. Pre/post test was done by all and feedback received by 80% of the training centers. Based upon the feedback of initial few training sessions, trainers arranged for flip charts, black boards, functional hub cutters and cold chain equipment before hand for next batches; LCD projector was used for immunization film and more focus was given to demonstration of safe injection practices; More emphasis was laid on practical demonstration of injection practices and the participants were taught on preparing list of due beneficiaries. 2. Knowledge and Skills of trainers in immunization training of health workers: 5 questions were asked to all the district trainers to demonstrate their training skills in areas of cold chain, injection administration, injection safety, AEFI, increasing immunization coverage and tracking of drop outs. Based on the responses, performance of each trainer was rated as excellent, good or average. Trainers performance was poor in AEFI management and tracking of drop outs in Bihar, Jharkhand, MP and UP. In other areas, it was variable. Orissa and Rajasthan performed better as compared to other states in all the parameters. (Table 7) Table 7: State wise % of trainers with excellent and good performance in training skills Indicators How would you describe and demonstrate "Conditioning of ice-packs" to HWs Bihar (n=6) Jhark hand (n=5) MP (n=6) Orissa (n=4) Rajast han (n=5) UP (n=12) UA (n=2) India (n=40) Demonstrate use of AD syringes for giving DPT injection to an infant Demonstrate use of Hub-cutter and safe waste disposal How did you train HWs on management of AEFIs How did you train HWs on "increasing immunization coverage" and tracking of dropouts Problems faced by the Trainers during training and Suggestions for future training 20% of the respondents cited no proper training facility as a major problem. Others informed about higher batch size, shortage of training materials, less time for training and lack of transport arrangements, etc. 29

30 Course contents were adequate and rationally developed according to 65% of respondents. One fifth felt that session on preparing due list of beneficiaries and VHNDs could be included. Training methodology was well designed and directed to the participants according to half of the respondents. One forth felt that more field based sessions with practical demonstration to be included. Facilities at the Training center: Proper training and transport facility at district level was required according to majority of the respondents. Training centre must be supported with adequate infrastructure. Number of Trainers: 3-4 trainers were adequate according to 2/3 rd of the respondents. 50% felt that few additional trainers should be trained to take care of absence of trainers. Funds must be made available for the rest of the batches, so that the remaining HW (M and F) are trained in the districts. Other support required: One sixth felt that training should be made more practical and extended to 3 days. Honorarium of the trainers must be increased. Training of Data Handlers and Cold Chain Handlers must be tagged with HW training. Refresher training must be planned. Training of AWW/ASHA must also be done along the same lines on RI. State level guidelines must be included in the ANM module locally Areas to be added in the Handbook: Few areas suggested for addition were details on VPDs, Communication skills, Details on newer vaccines, Refresher package, newer guidelines on RI, Vaccine management and Sessions on local formats. Areas to be deleted: None was the reply from all the respondents. Areas to be modified: None was the reply by half of the respondents. 10% suggested modifying the formats in local language. 2.5% suggested that Cold chain unit could be reduced and case studies added. 4. Facilities available at the Training Centers: 20% of the districts had no training facility. 80% of the districts had training centers and class rooms but only half had hostel rooms and transport facilities. There were interstate variations as shown in Table 8. Table 8: State wise percentage of training centers with available training facilities Training facilities Bihar (n=6) Jhark hand (n=5) MP (n=6) Orissa (n=4) Rajast han (n=5) UP (n=12) UA (n=2) No training facility available Classroom Hostel-rooms Transport India (n=40) 5. Opinion of Health Workers about the quality of Training Courses attended: 30

31 Duration of training: 56% of HWs found the duration of training to be adequate. Field visit organized: 68% of the health workers (10% in Jharkhand) reported that field visit was organized during training. Injections practiced: During field visits, only 38% of the health workers got opportunity to practice injections (12.5% in Bihar and 0 in Jharkhand)(Figure 15) Percent Figure 15 : Field Visit Organized Injections Practiced 68.3 India (n=313) Bihar (n=48) Jharkhand (n=40) MP (n=48) Orissa(n=32) 71.9 Rajasthan (n=39) UP (n=96) Uttarakhand (n=14) 30.8 Field Visit organized Injections practiced during field visits Immunization film was shown during training as reported by 64% of HWs (20% in Jharkhand). Referred the Immunization Handbook after training and availability at session site: 74% of HWs referred the Immunization Handbook after training but it was Percent Figure 16 : Referred Immunization Handbook after training and Handbook available at session site 73.6 India (n=313) 13.9 Bihar (n=48) Jharkhand (n=40) MP (n=48) 100 Orissa(n=32) Refer immunization handbook after training 46.9 Rajasthan (n=39) UP (n=96) 8.8 Uttarakhand (n=14) 61.5 Available at Session Site 0 31

32 only available at 14% of the session sites. (Figure 16) Sessions liked best: Demonstration and practice of injection techniques were liked by 1/3 rd of HWs. Other responses were watching the film on organizing and conducting vaccination sessions, sessions on National Immunization Schedule, cold chain, injection safety and waste-disposal. Sessions liked least: HWs could not recall any sessions which they did not like. Suggestions to improve the Immunization handbook: No specific suggestions were given to improve the Immunization handbook. V. few suggested for more pictures and diagrams. Boarding and lodging facilities at the training center: 1/3 rd of the HWs did not respond to the question. Almost 1/3 rd of the HWs reported that stay arrangements were either good or OK. 20% reported that no facilities were provided for stay during the training. 20% opted to stay at a relatives/ friends house. 4% reported that the facilities provided were not up to the mark. Difficulties faced during training: 90% of HWs did not face any difficulties. Only 6% complained that no arrangements for practical training were made. Need further training in immunization and the areas: 73% of HWs felt the need for further training in immunization. Areas for training were suggested as hands on practice of injection techniques by 1/4 th of the HWs; updates on vaccines, cold chain maintenance and microplanning by around 10% of respondents. Training methodology suggested: 70% HWs suggested field visits as the methodology; 51% suggested films; 36% role-plays while 30% group work and lectures. Suggestions for improving future training: 43% of the respondents suggested that future trainings be more practical oriented with field visits and hands-onpractice. Refresher training at least once a year was suggested by 17% of HWs. 11% wanted training for longer duration. 7% wanted more film shows and more use of audio-visual media during the training. D. The opinion of supervisors and beneficiaries about the performance and job behavior of health workers The supervisors of HWs at the PHC and the beneficiaries/caregivers were interviewed through house to house visits to assess the job behavior of the trained health workers. 1. Areas in which Supervisors noted change in performance of Health Workers after training 32

33 18% of respondents noted improved Tracking of drop-outs by HWs; 16% noted better recording and reporting; 15% noted better injection technique and safety; 7% noted improved immunization waste disposal; 4% improved knowledge of microplanning and 2.5% better cold chain maintenance and use of Hub Cutters. (Figure 17) Figure 17 : Supervisors noted change in the performance of HWs after training (n=82) Tracking of drop-outs Better recording and reporting Better injection technique and s afety Immunization was te dis posal improved knowledge of microplanning better cold cahin maintenance Use of Hub Cutters Knowledge of Care givers of children in the age group of 0-2 years: (Figure 18) 93% of the caregivers were aware about the place of immunization session; 3/4 th were reminded for vaccination prior to vaccination day and 72% knew about the site of vaccination of their child; 71% knew about minor adverse events following immunization and 70% had their children received age appropriate vaccines; 56% knew what to do in case minor adverse events following immunization occur and 49% knew when to go for the next due vaccine for their child. State wise comparisons are given in Table 9. Figure 18 : House to House visits Knowledge of Care givers (n=1231) Awarenes s about place of Immunization 93.2 Knowledge about child's vaccination site Knowledge about minor AEFIs Knowledge about handling minor AEFIs 55.7 Child received age appropriate vaccines 69.8 Knowsledge about when to go for next due vaccines Reminded for vaccination prior to vaccination day

34 Table 9: Knowledge of care givers during house to house visits (State wise %) Indicators Awareness about the place of immunization Knowledge about child s vaccination site Knowledge about minor AEFIs Knowledge of what to do in case minor AEFIs occur Child received age appropriate vaccines Knowledge about when to go for next due vaccines of the child Reminded for vaccination prior to vaccination day Bihar n=182 Jharkh and n=160 MP n=172 Orissa n=128 Rajast han n= 154 UP n=384 Uttara khand n=51 India n= E. Non-training issues that would enhance on the job performance with respect to immunization service provision Interview of the PHC medical officers and observation of RI services at the PHC brought out some important non training issues which influence the job performance of the health workers. 1. Routine Immunization services available at the PHC (Figure 19) Map of the catchment area was available in only 39% of the PHCs. Estimation of beneficiaries and logistics as a part of Microplan was done by 60.5% of PHCs. Roster of HWs was available at 88% of the PHCs. Alternate Vaccine Delivery plan was available in 60.5% of the PHCs (36.4% in MP). No specific reason was mentioned by majority for not having AVD plan. 5% cited lack of funds and 4% lack of awareness about this system. State wise comparisons are given in Table 10. Figure 19 : Routine Immunization Services available at the PHCs (n=82) Map of catchment area including all sub-centers and distances from vaccine storage point 39 Estimation of beneficiaries and logistics for current year 60.5 Roster of health workers 87.5 Alternate vaccine delivery plan to supply the vaccines and logistics to session sites? 60.5 Is Coverage monitoring chart/drop out chart displayed at the PHC Stock-outs or shortage (vaccines, syringes etc.) reported in last 3 months

35 2. Coverage monitoring chart/drop out chart was displayed only at 3.7% of the PHCs Table 10: Immunization services available at the PHC (State wise %) RI Services at PHC Bihar (n=12) Jharkh and (n=10) MP (n=12) Orissa( n=8) Rajast han (n=10) UP (n=24) UA (n=6) India (n=82) Map of catchment area including all sub-centers and distances from vaccine storage point Estimation of beneficiaries and logistics for current year Roster of health workers Alternate vaccine delivery plan to supply the vaccines and logistics to session sites? Coverage monitoring chart/drop out chart displayed at the PHC Stock-outs or shortage (vaccines, syringes etc.) reported in last 3 months Disposal pit used for immunization waste disposal Stock-outs or shortage (vaccines, syringes etc.) in last 3 months was reported by 37% of PHCs covering all the states. 15% reported stock out of BCG vaccine, 9% reported measles, 7% reported 0.1 ml ADS, 6% reported for DT, TT and Measles vaccines. Figure 20 : Methods used for disposal of sharps at the PHC 42.0% 3.7% 54.3% Disposal Pit Other Means No proper method 35

36 4. Methods used for disposal of disinfected sharps (cut needles, broken vials & ampoules) (Figure 20) Though disposal pits were used in 54% of PHCs, only 30% were practicing disinfection. No proper method was available for waste disposal in 42% of the PHCs. Burial or burning was practiced by 22% of the PHCs as elicited on asking open ended question. 5. Supervisory visits undertaken to SC/Session site during last 3 months: On an average, Medical officers conducted 11 visits (4/month) and other supervisors of HWs conducted 22 visits (7/month) in the last three months. But no records were available at the PHC to support the supervisory visits in 45% of the PHCs and Supervisory checklists/reports were available only at 27% of the PHCs. (Figure 21) Figure 21: Records available at the PHC to support the supervisory visits (n=82) Percent No Records Supervisory checklist/reports Movement Register Log book 6. Cold chain and logistics management at PHC Vaccines were delivered through Alternate Vaccine Delivery System in 71% of session sites (100% in Bihar and 95% in Jharkhand). Conditioned ice packs were available in the vaccine carrier at 84% of session sites and VVM was found in correct stage in 98% of session sites. Ice packs were conditioned correctly at 48% PHCs; temperature log books were correctly maintained at 64% PHCs; stock registers were maintained correctly at 72.5% of the PHCs. VVM was found in usable stage in 96.3% of PHCs. Alternate Vaccine Delivery System was working very well in Bihar and Jharkhand 7. Availability of cold chain and injection safety logistics at the PHC and session site Functional DFs were available at 80%; ILRs at 78%; thermometers at 91% and voltage stabilizers at 82% of PHCs; Indent forms were available in 51% of PHCs and supply vouchers in 63%of PHCs. Functional hub cutters were available in 54% (21% in UP and 36

37 25% in Bihar) and bleaching solution in 38% of PHCs. At the session site, availability of functional hub cutters (53%), red bags (36%) and black bags (32%) was poor. 8. Tracking tools available at the PHC and session site Immunization cards were available at 86% of session sites and 83% of PHCs, Immunization registers were available in 77% and Tracking bags in 49% of the PHCs. Coverage monitoring charts were available in only 3.7% of the PHCs (25% in Orissa). How monitoring and supervision improved the performance of HWs (Orissa - Success story) Three Govt. medical colleges were assigned the monitoring of HW training in addition to the monitoring of the Immunization program on a regional basis. Community Medicine faculties from these medical colleges were trained as the key master trainers. They in turn trained the district trainers at zonal level and only those trainers with good training skills were selected for training the HWs. 15% of the HW trainings were monitored and supportive supervision was provided. Each monitor (medical college faculty) stayed in the district at the training venue for entire 2 days of the training and wherever s/he identified severe gaps, s/he facilitated the session and provided supportive supervision to the district facilitators and visited again after 2-3 months when the same facilitators conducted the training. Same monitors also visited the field session sites, assessed the skills of health workers and gave appropriate feedback to the respective district trainers. Districts with poor performance received a letter directly from the medical college with a copy marked to NRHM-MD, Director FW and UNICEF. Mid course corrective actions based on monthly monitoring reports were undertaken at several places. Quarterly review meetings are held with all the monitors at the state level. The reports are shared and appropriate actions taken by the state through DPMU. 37

38 5. Major conclusions and Recommendations 1. Training All states with the exception of Jharkhand have trained more than 80% of female health workers (ANMs) on vaccination of children and pregnant women, while the newly recruited contractual female health workers and other categories as male health workers, health assistants, cold chain handlers, data handlers etc are still left to be trained for their specific tasks. State governments need to release funds to the districts to continue the in-service training for all health workers including HW (male), HA (male),ha (female), cold chain and data handlers in their specific tasks at PHC and District level as well as newly recruited contractual health workers. Sixty-five percent of trainers deemed the course contents were adequate and rationally developed. Suggested topics to be included in the Handbook were details on VPDs, communication techniques and details on newer vaccines. One in four of those interviewed felt that more field based sessions with practical demonstration of preparing due list of beneficiaries and VHNDs should be included. Training material should be updated based on recent policy changes and more emphasis is required on improving communication skills (in the handbook as well as in the facilitators guide). Eighty percent of the districts had training centers and class rooms available, however, only half had hostel rooms and transport facilities. Residential arrangements were made by 35% of the districts. There is need to establish or strengthen the logistical capacity and infrastructure of training facilities and residential arrangements need to be made in each district for training. The average batch size for HW training was 26, well above the norm of 15. During field visits, only 38% of the health workers reported having the opportunity to practice injections. While all training centers distributed Immunization Handbooks in local language, only 64% of HWs reported viewing the Immunization film during training. States need to enforce guidelines for a batch size of 15 and at least 3 trainers for each training course. Hands-on opportunities must be provided to the trainees by taking the participants in small batches to 3-4 different session sites. Greater oversight of training sessions by state and district officials as well as partners is necessary. Sixty percent of health workers reported that only 2 to 3 trainers were involved on both days of training. Trainers skills were found to be weak in conditioning of ice-packs, use of hubcutters, AEFI management and tracking of drop outs. Trainers from Orissa and Rajasthan performed better as compared to the other states on all parameters. States need to train more trainers to ensure adequate number of trainers in all the districts. Trainers skills need to be evaluated and strengthened through annual refresher trainings. 38

39 One district level officer should be designated as nodal officer for coordinating the immunization training of all health functionaries. 2. Performance of health workers after training in Immunization Injection administration skills of Health Workers appear to have improved since the PNA study. 85% of HWs were injecting vaccine using the correct route per antigen; 75% were allowing dose to self disperse instead of massaging and 67% were maintaining aseptic technique. 98% of the health workers were found competent in using ADS and 96% in using new disposable syringe for each reconstitution. However, only half of observed health workers cut each syringe with hubcutters immediately after use. Regular training and retraining of health workers in RI is required every year. There is a need to provide hands on training to health workers to practice injection administration techniques. Documentation (recording and reporting) skills of health workers also improved after training as compared to the baseline PNA study with 73% of all health workers documenting each vaccination correctly and completely. However, reporting of VPDs and AEFIs was found to be very poor across all the states. HWs need to be supervised to ensure that they prepare the due lists, fill counterfoils and update the Immunization registers regularly. HWs should be sensitized and encouraged for reporting of VPDs and AEFIs. Interpersonal Communication skills of health workers were found to be weak even after training. 44% were welcoming beneficiaries; 40% were explaining potential adverse events following immunization and 39% were discussing with beneficiaries/parents about the next visit. Only 18% were explaining what vaccines would be given and the VPDs prevented; 13% were screening the beneficiaries for contraindications and 15% were asking the beneficiaries to wait for minutes after vaccination. Need to improve communication skills and technical knowledge of the health workers by ensuring supportive supervision and on the job training by MO/PHC during immunization sessions. As a constant reminder for health functionaries, key IPC messages in local language should be displayed at the session site. While 93% of beneficiaries/caregivers knew about the place of immunization session, only 49% caregivers knew when to go for next due vaccine. Three-fourths of care givers were reminded for vaccination prior to vaccination day by the social mobilizer and 70% children had received age appropriate vaccines. IEC and IPC in the community need to be improved. HWs need to share the due list of beneficiaries with ASHA/AWW for tracking of drop-outs. ASHAs need to be trained for better community mobilization especially in the resistant group. 3. Immunization Program support to HWs Coverage / Microplanning 39

40 Eighty six percent of the sessions were being held as per the RI microplans; however, a map of the catchment area was available in only 39% of the PHCs. Estimation of number of beneficiaries and logistics as a part of microplan was limited (61% of PHCs). Though roster of HWs was available at 88% of the PHCs, the Alternate Vaccine Delivery plan was available in only 61% of the PHCs. Proper Microplans with maps are required so that no area is missed. Maps showing the subcentres, distance of session sites from ILR points and alternate vaccine delivery plan need to be displayed at each PHC. Cold chain and logistics management Designated and trained cold chain handlers were not available at the PHCs. Vaccines were delivered through Alternate Vaccine Delivery System to 71 % of session sites. 73% of PHCs were correctly maintaining the stock registers; 64% were correctly maintaining the temperature log books and only 48% were conditioning the icepacks correctly. Stock-outs or shortage of vaccines or syringes in last 3 months were reported by 37% of PHCs covering all the states. Twice daily recording of temperature in the log books needs to be monitored. Urgently designate and train the cold chain handlers in their specific tasks. Nonfunctioning cold chain equipment needs to be repaired or replaced. Ensure regular supplies of immunization cards, registers, tracking bags and coverage monitoring charts. Injection safety and waste disposal Forty nine percent HWs were cutting AD syringes immediately after use and 35% were using red bags to keep cut AD syringes. Availability of functional hub cutters at PHC (54%) and Session site (53%) was poor. Availability of red bags (36%) and black bags (32%) was also poor at session site. Though disposal pits were made by 54% of PHCs, disinfection was practiced only by 30% PHCs; syringes and needles were thrown into these pits meant for sharps only; burning and burying of the immunization waste was also practiced. Only 68% of the trainers had good knowledge of the use of hub cutters and safe disposal of immunization waste. States need to establish a system of collection of segregated waste from session site to PHC for disinfection and proper disposal; ensure regular supplies of hubcutters, bleaching solution, red and black bags and also construction and proper utilization of waste pits for disposal of sharps. Monitoring and supervision by medical officers needs to be ensured. AEFIs Only 53% of the trainers of health workers had good knowledge on the management of AEFIs and 69% of the health workers knew proper treatment of minor AEFIs. However, only 40% of health workers explained potential AEFIs to care givers. Only 7% of the PHCs had reported AEFIs in the MPRs of last three months preceding the study. Training surrounding all aspects of AEFI needs to be strengthened to ensure that medical officers regularly sensitize the health workers on reporting and management of AEFIs and that HWs explain potential AEFIs and their management consistently to each beneficiary / caregiver. Tracking and mobilization of beneficiaries 40

41 During observed sessions, only 39% of health workers discussed with beneficiaries /parents the date of the next visit. Approximately 30% of beneficiaries did not receive age-appropriate vaccinations. Only 4% PHCs displayed coverage monitoring charts and only 50% of trainers had adequate knowledge about tracking of dropouts. Preparing due lists, updating counterfoils and using tracking bags should be standardized for tracking beneficiaries. HWs need to update immunization register by including information from the records of AWW/ASHA as well as new born tracking booklets of SIAs. Coverage monitoring charts should be displayed in every PHC and sub center and this should be ensured through supportive supervision and on the job training by MO/PHC during immunization sessions. Monitoring and Supervision Though medical officers of the PHC reported monitoring on an average 4 session sites per month and other supervisors monitored around 7 session sites per month during last 3 months, no records to support the supervisory visits were available in 45% of the PHCs. Fifty six percent of health workers reported that MO had visited their session sites in the last 3 months while 63% reported the visit of other supervisors. Provide mobility support to medical officers for the field visit and supervision. Train all supervisors on monitoring and supervision of all the health functionaries involved in immunization services. The monitoring by BLOCK TEAM and by DISTRICT TEAM should be made mandatory; this needs specific guidelines and tools to be prepared and shared with all states and districts. Way Forward Update training materials at national level for training the health workers, cold chain and data handlers in the states. Use innovative training methodologies e.g. developing web based training packages for e- learning, videos etc. States need to provide refresher training to the trainers as well as all health functionaries every year and orientation/induction training for the new staff. Annual State PIPs under NRHM need to include all the immunization training related activities. Regular monitoring to understand weaknesses and tailored supportive supervision of all health functionaries is required to reinforce training and improve quality of services and ultimately immunization coverage rates. 41

42 Annex 1: State wise summary of observations and suggestions by the study teams 1. Bihar (Buxar, Champaran East, Gaya, Darbhanga, Kishanganj, Khagaria) Areas Major Observations Recommendations Quality of Training 1. Funds not received for last one year to train the backlog of contractual and newly recruited Health Workers 2. Residential arrangements were made only in 2 out of 6 districts 3. Training facility was available in 50% of the districts 4. Hands on skill practice was not provided at any of the districts 5. Training batch size was of participants, 2 batches trained together at some places 6. Only 3 district trainers were available in some districts. 1. Need to establish or strengthen the training facilities for providing regular in-service training to the health service providers in each district. 2. Pre-Training venue assessment must be done by the state/district to ensure proper venue site/arrangement. Gaps noted during the exercise must be addressed by the State / Districts 3. Funds with revised guidelines for batch size of 15 HWs to be trained by 3 trainers/batch need to be sent from the state to the districts 4. Residential arrangements need to be made in each district for training. 5. Hands on skill practice opportunity must be provided to the trainees in small batches. 6. Training of more trainers in each district to be considered. 7. Training of cold chain and data handlers, male HW and HA, left over ANMs and LHVs is required. 8. District Level Monitoring of the training sessions must be done in future training programmes with partners help 42

43 Areas Major Observations Recommendations Immunization Program support to HWs Performance of HWs Community response 1. Excellent functioning Alternate Vaccine Delivery system, adequately staffed sub centers and PHCs, Medical officers conduct monitoring of Muskan sessions at some places. 2. Map of the area with sub-centers and AVD plan was not available at majority of the PHCs. 3. Nonfunctional and Condemnable cold chain equipment and vehicles were lying in the districts and blocks. 4. Acute shortage of functional cold chain equipment, hub-cutters, thermometers and immunization cards 5. Poor maintenance of records for cold chain and logistics. Temperature log books were not available. 6. Cold chain handler not designated and not trained at most of the blocks. 7. Poor waste management-no hubcutters, bleaching solution and immunization cards. All syringes and needles were put into the disposal pit. 8. Poor Supervision and Monitoring by District and Block Officials. No records of supervision by MO were available. 9. No reporting of AEFIs at any of the blocks 1. Majority of HWs were competent. Sufficient numbers available for social mobilization. 2. Due list of beneficiaries was not available. 3. Map of the areas were not displayed. 4. Key IPC messages were not given. 5. Poor tracking of beneficiaries. 6. No Immunization cards and hubcutters available. 7. Waste disposal was not as per guidelines. 8. Injection administration skills and screening for contraindications need improvement. 1. Community is very receptive and acceptance of RI services is good 2. Dependent upon AWW/ASHA for the vaccination 3. Majority did not know about next date of vaccination 1. Improve micro-planning 2. Condemnation and replacement of non functional cold chain equipment is urgently required. 3. Supply hubcutters, cold chain equipment and immunization cards. 4. Improve waste management by proper monitoring from district level. 5. District and Block Level Supervision needs improvement. 6. Need to train the medical officers, cold chain handlers and data handlers 7. Sensitization of health workers on reporting of VPDs and AEFIs is required. 8. Training of all categories of supervisors is required on monitoring and supervision of health functionaries involved in immunization services. 1. Provide supervision and on the job training to HWs sp. For micro-planning, vaccination techniques, IPC, tracking and recording and reporting etc. 2. Train HWs to prepare due list of beneficiaries for every session. 3. Provide immunization cards, PCM, functional hubcutters, red and black bags and tracking bags to the HWs. 4. Ensure HWs give key IPC messages to the beneficiaries display such a poster in Hindi at each session site. 1. ANM, in coordination with AWW and ASHA, needs to improve IPC and tracking of beneficiaries. 2. ASHAs need to be trained for better community mobilization. 43

44 2, Jharkhand (Bokaro, Gumla, Hazaribagh, Godda, Palamu) Areas Major Observations Recommendations Quality of Training Immunization Program support to HWs Performance of HWs 1. Trainings were held at block PHCs by MOs trained at District level. 2. Field visits to practice injection techniques were not held and hands on skills practice were not provided. 3. Contractual ANMs were not included in the training load initially and no budget was available with the district for training of LHVs. 4. The recall of the trainers regarding the training and the Handbook was poor. 5. Training guidelines were not followed; Handbooks not distributed and Immunization film was not shown at some places 1. Alt vaccine delivery system was functioning well. 2. Microplan with all components was not in place 3. Immunization waste disposal was very poor. Concept of Safety pits not as per laid down GoI guidelines 4. Poor supervision of RI sessions by MOs. 5. Solar cold chain equipment was lying dysfunctional. Much equipment was lying in irreparable condition. 6. Cold chain handlers lacked adequate knowledge. 7. Many of the ANMS were not paid their salaries for months. 1. Knowledge of ANMs on RI was very good. ANMs were motivated. ASHAs were giving good support to HWs. Due beneficiaries list was found at session sites. Despite inadequate supervision, they were working well 2. Immunization waste disposal was not proper. 3. Recapping of needles was done at some places. 4. Four key messages were not given uniformly. 5. MCH register was very heavy and difficult to carry. 6. Ticklers bags pockets size was smaller than card size. 7. Inadequate recording and reporting at session site. 8. Injection administration skills and screening for contraindications need improvement. 1. Training must be conducted at a well equipped district training center. 2. Field Visit should be conducted during training. 3. Opportunity for hands-on skill practice must be provided to all the participants. 4. Monitoring to ensure that the training guidelines are followed, must be conducted 5. Training of more trainers in each district needs to be considered. 6. Funds should be released from the state to train the backlog of health workers. 1. Prepare area maps and microplans. 2. Solar equipment can either be repaired or modified and used on electricity. 3. Irreparable equipment to be condemned. 4. ILRs and DFs must be installed in the blocks on priority. 5. Teach proper disposal of Immunization waste. Supply twin buckets. Safety Pits must be constructed as per laid down GoI guidelines. 6. Train cold chain handlers 1. Orientation on waste management needs to be done. 2. ANMs to be encouraged to give 4 key messages. 3. Registers need to be lighter and smaller. 4. New tickler bags with right size to be provided. 5. Proper training of all ANMs, LHVs, Cold chain handlers and MOs is must at District level. 6. Hands on practice are required at District level under the supervision of District trainers to improve the injection administration skills and tracking of beneficiaries. 44

45 Areas Major Observations Community response 1. Aware about the immunization site. Minimal resistance. 2. Majority of interviewed beneficiaries had received vaccines as per schedule 3. AWW or Sahiya need to mobilize them for every session. 4. Many are passive receivers of services. If the card is lost or torn, they delay the next dose or drop out altogether. Recommendations 1. Four key messages by ANM on regular basis can bring change. 2. IEC activity from district or state level regarding RI will help. 3. MP (Ratlam, Bhopal, Vidisha, Indore, Betul and Anuppur) Areas Major Observations Recommendations Quality of Training Immunization Program support to HWs 1. Contractual ANMs were not included in the training load initially 2. State Level instructions were provided for training of HW(F) only 3. Training institutions with proper residential facility were not available in half of the districts. 4. Inadequate funds were made available 5. No field visits were conducted during the training resulting in lack of hands on practice by the HWs. 6. Inadequate no of trainers and mismatch between recorded and actual no of trained trainers. 1. Microplans and cold chain equipment were available 2. Alternate Vaccine Delivery System was not functional, 3. Plan for supervision and monitoring of RI sessions was not prepared. 4. Supervision by medical officers / supervisors was poor / ineffective 5. Poor cold chain maintenance and temperature recording, 6. No dedicated cold chain handlers were available. They were doing other accessory jobs in the PHCs 7. Poor record keeping and waste disposal. Vaccine Stock Registers were not properly maintained 8. Medical Officers were not oriented on recent initiatives on RI 1. Health Workers other than ANM must also be trained in RI 2. Proper training facilities with residential arrangements may be identified in all districts. 3. Timely provision of funds, 4. More District Level trainers must be trained so as to continue training 5. District Health Educator must be a part of the team to train on social issues related to RI. 6. Hands on practice should be regularly used. 7. All training material must be available at the training facility before hand 1. AVD to be made functional. 2. Microplanning must be done on priority, buffer stocks to be kept to support RI activity 3. Train cold chain handlers, medical officers and all other staff. 4. Need for close supervision and monitoring by district level; avoid missing of sessions. 5. Supply hubcutters, red and black bags for proper waste management. 6. Training of all categories of supervisors is required on monitoring and supervision of health functionaries involved in immunization services. 45

46 Areas Major Observations Recommendations Performance of HWs 1. Sessions being held as per RI Microplan. Health staff and supportive staff were present. 2. Inadequate social mobilization. 3. Poor injection technique. 4. Poor communication with parents. 5. Poor recording and updating of counterfoils. 6. Basic facilities at session site (table, chairs) not available. 1. Retraining of HWs by providing hands on practice for improving aseptic Injection technique, Communication skills and Social Mobilization. 2. Provision of basic facilities at session site. 3. Improve tracking of beneficiaries by updating of the counterfoils and use of tracking bags. Community response 1. Cooperative and receptive to advice. 2. Unaware about the services and poor knowledge of benefits of immunization. 1. ASHA & AWW can be involved in educating parents at session sites. 2. IEC materials to be displayed at session site. 46

47 4. Orissa (Balasore, Gajapati, Nayagarh, Nuapada) Areas Major Observations Recommendations Quality of Training Immunization Program support to HWs Performance of HWs Community response 1. Training was residential. 2. Training was of good quality in Nayagarh and Nuapada 3. Newly Recruited ANMs were not trained 4. Male health workers and health assistants were not trained at some places 5. No budget available with the districts for training 1. Good Support staff available at the Block PHC. Most of the HWs were trained. AVD was in practice. Good record keeping and documentation practiced at the PHC 2. Blocks were seen using stock pass books for receiving and distributing vaccines 3. Maps were not displayed. 4. No Proper Disinfection & Disposal of used AD- Syringes at the PHC. Safety Pits were constructed but not used. 5. Block Level Supervision of the RI by medical officers was poor. Sickness rate of DFs and ILRs was high. 6. Vaccine Management in terms of buffer stock needs to be addressed 1. Majority were knowledgeable and motivated. 2. Good record keeping in the Vaccine pass book & proper vaccine indenting was practiced by health Workers. 3. Good team work with Link workers like ASHA & AWW. 4. Duration of RI-sessions held was short i.e. (From 9 am to 12 noon). 5. Only one RI day practiced in the State. 6. No proper waste disposal practiced. 7. Screening for contraindications needs to be improved. 1. Most of them were aware of antigens, completed age appropriate Immunization as per the RI cards available and were aware about mobilization by link workers. 2. Card retention rate was less, some beneficiaries were not aware about the next due date of vaccination. 1. Male health workers, health assistants and cold chain handlers need to be trained. 2. Need to improve the hostel facilities at some places. 3. Supervision and monitoring of the training sessions must improve 1. Improve micro plans with maps. 2. Increase immunization sessions to two per week. 3. Streamline immunization waste disposal system. 4. Cold chain handlers training at the blocks is needed. 5. Training of male Health Workers needs to be done using the same module. 6. Supervision of RI sessions by Govt. officials must improve. MOs need to supervise Immunization sessions. 7. Vaccine management and Cold Chain issues must be addressed at District level. 1. RI-session should be held from 9AM to 4 PM RI days will help to conduct more sessions for other hard to reach villages. 3. Refresher training is needed every alternate year. 4. Improve biomedical waste disposal practices. 5. Ensure regular supply of immunization cards and proper filling up of counterfoils during the sessions. 1. Outreach sessions should be planned more frequently, at least twice a week. 2. Caregivers should be given the exact due date for the next visit 47

48 5. Rajasthan (Jaisalmer, Dholpur, Pali, Bikaner, Bhilwara) Areas Major Observations Recommendations Quality of Training 1. Residential arrangements were available in 2 out of 5 districts. 2. Field visit was conducted in places where the beneficiary load was not sufficient 3. Inadequate funds were reported for field visit of the trainees for hands on practice. 4. Inadequate no. of trainers in some districts 5. Knowledge gap among trainers was observed. 1. Proper guidelines from the state to the districts regarding budget, stay and food arrangements should be sent. 2. Ensure hands on training for the HWs. 3. Build a pool of trainers in each district, sp. from medical officers. 4. RI training must not be clubbed with any other training. Immunization Program support to HWs 1. Good infrastructure at block level. Most of ANMs were trained. Tracking bags & Hub cutters were provided to all ANMs, AVD is established by PHC in decentralized manner, every PHC has cold chain equipment. 2. ANM roster was available but No Maps, List of villages was available but list of Dhani & Mazhra was not available. 3. Block LHV/Cold chain handler/data handler were not trained. Vaccine Management needs improvement 4. Poor immunization waste management. Open safety pits. Sharps with other waste of session sites were dumped in the safety pits. 5. Lack of supervision of the session sites by Medical officers of the block 1. Proper microplanning required with maps indicating distances from cold chain storage point; 2. Improvement in record keeping especially updating of logistics, issue registers, indent forms, temperature log books; 3. Immunization waste management to be streamlined. 4. Train the Cold Chain handlers, Block LHV and Medical Officers on RI. 5. Ensure that vaccines are placed in zipper bags and are delivered on the session day. 6. Supervision by district and block medical officers must improve 48

49 Areas Major Observations Recommendations Performance of HWs Communit y response 1. ANMs maintained good co-ordination with anganwadi workers and ASHA for mobilizing community, made list of due beneficiaries. 2. Immunization registers were available at most of the sites 3. Knowledge of ANMs was good at some places average at others 4. Technique of injection was not correct; DPT was given on Hip and Measles at forearm at some sites. 5. Functional hub cutter were not there at many sites, even if present ANM was not cutting the needle immediately after injection. 6. Recapping of needles was practiced by some workers. 7. Record keeping was not proper; ANMs were using daily dairy and not Immunization registers. 8. Tracking bags even if available, ANM was not knowing how to use it, tally sheet was not used. 9. ANMs had no concept of waste management; all were burning the syringes even if they were provided with safety boxes. 10. Vaccine carrier was brought a day before in Jaisalmer, resulting in melting of ice-packs key IPC messages were not given. 1. Community was aware and receptive 2. Not aware about importance of RI card, benefits of immunization and possible AEFIs. 3. Some resistance among migratory population. 1. Hands on practice needed as majority of health workers did not have field visits during the training. 2. Regular re-training to refresh and update the knowledge. 3. More emphasis on immunization techniques, biomedical waste management and tracking of drop outs. 4. Provision of good quality functional hubcutters, tracking bags, red & back bags etc. 5. All beneficiaries need to be screened for contraindications and asked to wait for minutes after vaccination. 6. Ensure that ANMs give 4 key IPC messages and update the counterfoils after vaccination. 7. Supportive supervision by medical officers is needed. 1. IEC and IPC in the community need to be improved. 2. Training of ASHA to learn better community mobilization especially in the resistant group. 49

50 Quality of Training 6. UP (Rampur, Moradabad, Lucknow, Saharanpur, Badaun, Sonbhadra, Ambedkarnagar, Basti, Fatehpur, Jaunpur, Mahoba) Areas Major Observations Recommendations 1. Training centers were available in 80% of the 1. Strengthen the training facilities including stay districts but stay arrangements were made only and transport in each district. by 40% of the districts. 2. Provide opportunity for hands on practice by 2. Less number of ToT trained trainers were taking the participants to 3-4 different session available in many of the districts. sites. 3. Batch size was found higher at some places 3. Train the backlog of HWs and other categories 4. No hands on skill practice were provided at as HW male, HA male, cold chain and data some places. handlers at PHC and District level. 5. Only ANMs and LHVs were trained. 4. Train more trainers for the districts with 6. Funds were not provided to complete the shortage. training. 5. Provide sufficient funds for training. 7. Frequent change of DIOs and transfer of TOT 6. DIO should be made responsible for only trained trainers at some places immunization and not for other programmes 7. Programme Managers at all levels must be involved in the training sessions 8. Monitoring of Training sessions by District Level Immunization Program support to HWs 1. Good Infrastructure including cold chain equipment available in all the blocks. Alternate vaccine delivery system is functioning in some districts. Vaccine supply is good. 2. Map of the area with sub-centers not available. 3. Alternate vaccine delivery not practiced in all the districts. 4. Cold chain equipment maintenance is poor. 5. Shortage of Immunization cards, functional hubcutters, red and black bags not available. 6. Poor waste management: shallow and open pits used for burning and burying the immunization waste. 7. No supervision is practiced. 8. No AEFI or VPD were reported in last 3 months 9. Huge no. of non-functional cold chain equipment were occupying space at the blocks. Officers and partner agencies must be done 1. Train Medical officers in immunization and urgently train the IO and ICC with HWs. 2. Map of the area showing sub-centers, distance from ILR points and alternate vaccine delivery plan need to be displayed at each PHC. 3. Provide mobility support to medical officers for the field visit and supervision of RI. 4. Train the staff in record maintenance, cold chain maintenance and safe waste disposal during meetings at the block. 5. Improve waste disposal by supplying hubcutters, using disinfection procedures and monitoring 6. Cold Chain Management and supply and stock management needs improvement. Recording of temperature and log books needs to be streamlined 7. Supply Immunization Cards and voltage stabilizer. 50

51 Areas Major Observations Recommendations Performance of HWs Community response 1. HWs have improved knowledge and techniques of vaccination after training, conducting sessions as per micro-plan. 2. Due list of beneficiaries was not available at the session site. 3. Some ANMs still found practicing unsafe injection practices as recapping needles. 4. Key IPC messages were not given. 5. Poor documentation of vaccination and updating of counterfoils. 6. Poor social mobilization. 7. Improper waste disposal, functional hub cutters, red and black bags were not available. 8. Screening for contraindications and asking beneficiaries to wait for minutes needs to be improved. 9. Sessions were being held at the same site in a big village. 1. Good acceptance and awareness in the community 2. Majority did not have immunization cards. 3. Not aware about the next due date. 4. Dependent on the ASHA and AWW to take them to the site. 5. Poor knowledge about AEFIs and their management. 1. Need for hands on training and practice of injection administration techniques. 2. Regular training and retraining in RI every year. 3. Proper microplanning is required with map so that no area is missed. 4. Supervision and OJT by MOs for use of tracking bags, IPC skills. 5. Poster on key messages in Hindi to be displayed at the session site. 6. Provide logistics of PCM, Hubcutters, red bags, tally sheets etc. 7. Need to update immunization register by including information from the registers of AWW and ASHA as well as New born tracking booklets of SIAs. 8. Preparing due lists, updating counterfoils and using tracking bags should be encouraged. 9. Different session sites can be planned in big villages. 1. Good Quality IPC by health Workers, AWWs and ASHAs will motivate mothers to come for RI, at the Session sites. 2. To utilize ASHA effectively for tracking of new born and drop-outs. 51

52 Uttarakhand (Tehri, Champawat) Areas Major Observations Recommendations Quality of Training Immunization Program support to HWs 1. Newly Recruited and contractual ANMs were not included in the original training plan 2. Training of HW (M) was not planned for training, as HW (M) is attached to Panchayat Department 3. No Training facility in the districts - training done at CHC and PHC. 4. Improper stay arrangements at the block level. 5. Hands on training were not conducted. 6. Only 2 trainers were trained. 7. Gaps were found in trainer's knowledge on cold chain, micro-planning, waste disposal etc. 8. Inadequate and unorganized records of training 9. Pharmacists also need to be trained in RI 1. Good infrastructure available, good electricity supply and cold chain equipment in good condition 2. Vacant sub-centers and male HWs were attached to Panchayat dept. with minimal involvement in health related activities. 3. Map of the area with sub-centers not available. No Microplan only VHND roster was available. 4. No alternate vaccine delivery practiced, vaccine being given to ANMs one or two days prior to immunization day. 5. No proper method for waste disposal-no bleaching solution, open disposal pit-burning and burying done. 6. No AEFI or VPD reported in last 3 months. 7. No dedicated and trained Cold Chain Handler at Block 1. Training center with all required training facilities including stay arrangements needs to be established in the district. 2. Refresher for half a day practical session at CHC for all ANMs in batches can be conducted. 3. POL/mobility support to district trainers to go to remote blocks be given. 4. Proper maintenance of training records by responsible persons is needed. 5. Cold Chain Handler basic training needs to be done. 1. Proper utilization of male HW, pharmacist, staff nurse after training. 2. Vaccine flow from District level needs to be addressed. Courier system can be used for vaccine delivery. 3. Preparing Microplans on priority. Display map of the area showing sub-centers, distance from ILR points and alternate vaccine delivery plan. The missed villages can be covered using 3 Wednesdays. 4. Train cold chain handlers and data handlers. 5. Proper waste disposal as per CPCB guidelines. 6. Train Medical Officers on RI 52

53 Areas Major Observations Recommendations 1. Good communication with ASHA/AWW, Motivated health staff, Regular visits to VHNDs, Good supply of logistics to HWs, conducting sessions on all Saturdays in AWCs. 2. Due list of beneficiaries and counterfoils were not available. 3. Injection technique was faulty in majority of sessions. 4. Key IPC messages not given. 5. Poor documentation of vaccination. 6. No hub-cutters, no plastic bags available 7. Screening for contraindications was a weak area. Performance of HWs Community response 1. Community awareness about place of session (sub center) on first Wednesday was high. 2. Demand generation was good, almost no resistance. 3. Accessibility is an issue as population is scattered, terrain is difficult and sessions are not held in all the villages. 1. Regular training of HWs to practice the injection administration skills is needed. 2. Supervision and OJT by MOs. 3. Tracking bags were distributed; their use should be explained and ensured. 4. Poster on key messages in Hindi to be displayed at the session site. 1. Sessions can be held once in 3 months in very difficult to reach and uncovered villages. 2. Need to keep in touch with AWW to know the date of vaccination and to keep the immunization cards safe 53

54 Annex-2: Suggested guidelines for immunization training of health workers Duration of training No. of trainees per batch Venue Trainers No. of trainers Methodology 2 working days District Hospital and ANM Training Center (ANMTC)/ First Referral Unit/ Community Health Center (CHC) District Immunization Officer /ANMTC trainer / Pediatrician/ CHC Medical Officer 1 facilitator for each group of 4-5 trainees Group discussions, Exercises, Demonstration and Return Demonstration, Hands-onpractice, Role play, Field Visit, Film show Annex-3: Tentative Programme for Immunization Training of Health Workers Day-I Registration Inauguration, Expectations of the participants and Pre testing Introduction and formation of groups of 4-5 participants with one Unit -1 facilitator each Briefing on VPDs and Vaccines followed by film. Unit Tea Quiz on filling of National Immunization Schedule in groups Unit Lunch Discussion on microplan for immunization Unit -6 Preparation of microplan (exercise). Discussion on session site checklist. Conducting immunization session and educating parents (role play) Tea Briefing on cold chain and injection safety equipment and Records and Reports Units- 4, 5 and 8 Day-II Field visit Each group to visit different PHC/CHC/Distt Hospital to observe cold chain system and practice giving safe injections and dispose immunization waste safely as per the guidelines using hub cutter and Demonstration on records and reports Lunch Discussions on observations made during the field visit and Each group to present Discussion on AEFI and how to prevent it: Unit Tea Discussion on surveillance of VPDs: Unit Discussion on how to involve community for increasing coverage and Unit-9 reducing dropout Open discussion, Post test and conclusion. 54

55 Annex-4: Detailed day-wise Tasks for each team No State Session Day 1st day 2nd day 3rd day 4th day 5th day 1 Bihar Wednesday, Friday 2 Jharkhand Thursday, Saturday 3 Madhya Pradesh Tuesday, Friday Tuesday District visit Wednesday District visit Monday District interviews 4 Orissa Wednesday Tuesday visit to District and 2 PHCs 5 Rajasthan Thursday, Monday 6 Uttar Pradesh Wednesday, Saturday 7 Uttarakhand Wednesday, Saturday Wednesday District visit Tuesday District visit Tuesday District visit Visit to 4 session sites Visit to 4 session sites Visit to 4 session sites Visit to 8 session sites Visit to 4 session sites Visit to 4 session sites Visit to 4 session sites Interviews at PHCs (both) Interviews at PHCs (both) Interviews at PHC -1 Visit to 4 session sites Visit to 4 session sites Interviews at PHC Visit to 4 session sites Interviews at PHC-1 Interviews at PHC-1 Interviews at PHC-1 Interviews at PHC-2 Interviews at PHC-2 Interviews at PHC-2 Monday visit to 4 Session sites Visit to 4 session sites Visit to 4 session sites Day 1 (All states) Location District HQ Selected Block 1 (and block 2 for Orissa) Tasks Meet the CMO / CS. Explain the purpose and plan of the visit Meet the DIO and district trainers. Arrange for one vehicle for each day and two vehicles for the two session days. Visit District training center and collect information in study tool-1 Collect the list of health workers trained from the selected blocks Select two Blocks randomly (with trained health workers available) - one within15 kms and other more than 15 kms from district HQ; Collect the Monthly Progress Report for last 3 months for the selected blocks Meet the MO. Explain the purpose and plan of the visit Take copy of the micro-plan of the PHC, Map of the block and list of trained health workers. Divide the Block into 4 zones and select one trained health worker from each zone randomly (total four health workers) for observation on session day. Exclude the HWs posted at the PHC HQ. Request MO for 1 person to accompany each team member to the session site. Note the mobile number of the medical officer before leaving for the session site If possible try to fill study tools 2 and 3. 55

56 Day 2 (All states) Day 3 (UP, Uttarakhand, MP and Rajasthan) Day 3 (Bihar, Jharkhand) Day 4 (UP, Uttarakhand, MP and Rajasthan) Day 4 (Bihar, Jharkhand) Day 5 (UP, Uttarakhand, MP and Rajasthan) 4 Session sites of Block 1 (and block 2 for Orissa) Selected Block 1 Selected Blocks 1 and 2 Selected Block 2 4 Session sites of Block 2 4 Session sites of Block 2 Each one of you will visit two health workers (at different session sites) and explain the purpose and plan of your visit Collect information in study tool-4 by talking to health worker and observation; study tool-5 by observing the HW providing services to 2 beneficiaries and exit interview of two care givers and study tool-6 by visiting 4 houses with 0-2 yrs children and talking to their caregivers. Collect information in study tool-2 and 3 Enter the data into the Data entry tool in excel format as provided. Collect information in study tool-2 and 3 Select four trained health workers for observation on session day Collect information in study tool-2 and 3 Select four trained health workers for observation on session day Same as given above against day2 for all states. Same as given above against day2 for all states. 56

57 Annex-5: Study Tools 1 to 6 Study Tool No. 1 Tool to collect information from District Immunization Officer/ District Training Coordinator/Trainer A. General Information 1 State (name ): 2 District (name ): 3 Date of interview (date on which this format is filled) 4 Name/s of Investigators Name of the District Immunization Officer (DIO) 6 Name and designation of the interviewee/s B. Information regarding training of health workers in the district (ask specifically for two days training with Immunization Handbook for Health Workers) 1. Routine Immunization Training status of health workers in the District (write the numbers) Category Staff in position (number) ANM(HW-F) LHV (HA-F) HW (M) HA (M) Any other category (specify) Total (Tick the responses where required) 2 No. of training courses (batches) conducted 3 When was last training batch conducted Staff Received training (number) Reasons if all not trained with in 1 yr 1-2 yrs back >2 yrs back 4 No. of TOT trained trainers available in the district (trained as trainers at state / divisional level) 5 No. of Trainers involved in each HW training course on >4 both days (check report if available) 6 Training was residential (trainees stayed overnight) Yes No 7 Field visit organized to DH/CHC/PHC for the trainees to Yes No practice giving safe injections (ask where?) 8 Training and other materials given to all participants (in local language) (First ask open ended question. Prompt only if unable to answer) 9 Pre/post test done. If yes, try to verify (ask for few filled in Handout no. 1) 10 Feedback received from trainees at the end of training (ask for a few filled in Handout no. 8) Immunization Handbook Handouts from Facilitators Guide Certificates Other (specify) Yes No Yes No 57

58 11 Mention any specific action taken to improve training based on feedback. C. Ask the following questions to assess the knowledge and skills of the trainers and grade the responses Questions Expected Answers Rating 1 How would you describe and demonstrate conditioning of icepacks to HWs 2 Demonstrate use of AD syringe for giving DPT injection to an infant 1. Remove the ice packs from the freezer and keep them outside till you hear the sound of water inside the icepack when shaken next to the ear. 2. The icepacks need to sweat, i.e. some condensation or droplets of water on them. 3. The time taken for conditioning ice- packs is not fixed; it varies depending on the outside temperature. 1. Opens the package from the plunger side and removes the syringe by holding the barrel. 2. Site of injection Antero-lateral aspect of the thigh (mid-outer thigh). 3. Angle of injection - Hold the syringe like a pen in the right hand and push the needle straight down at 90 deg. through the skin. Penetrate deep into the muscle. Excellent (if all three points are mentioned) Good (If any two points mentioned) Average (if any one point mentioned) No Response (if no points mentioned) Excellent (if all three points are mentioned) Good (If any two points mentioned) Average (if any one point mentioned) No Response (if no points mentioned) 3 Demonstrate use of Hub-cutter and safe waste disposal 4 How did you train HWs on management of AEFIs 5 How did you train HWs on Increasing 1. Cut plastic hub of AD syringe and not the metal part of needle immediately after administering the injection at the immunization site using the Hub cutter 2. Treat the collected material in an autoclave or boil such waste in water for at least 10 minutes or chemical treatment (using at least 1% solution of sodium hypochlorite for 30 minutes). 3. Dispose the disinfected waste as follows: 1. Dispose the needles and broken vials in a safety pit/tank 2. Send the syringes and unbroken vials for recycling or landfill. 1. Discussed with participants any AEFIs they may have come across 2. Read definition and types of AEFI from the Handbook. 3. Discussed ways to minimize AEFIs in their areas 4. Discussed what to do if AEFI occurs. 1. Discussed possible reasons for the left-outs, drop-outs and fully immunized with trainees Excellent (if all three points are mentioned) Good (If any two points mentioned) Average (if any one point mentioned) No Response (if no points mentioned) Excellent (if all four points are mentioned) Good (If any three points mentioned) Average (if any one or two point mentioned) No Response (if no points mentioned) Excellent (if all three points are mentioned) 58

59 Immunization Coverage and tracking of drop outs? by dividing them in 3 groups, each with one facilitator 2. Discussed possible ways of addressing dropouts and left outs based on Actions to be taken mentioned in the Handbook and their field situations. 3. Showed film on Improving Immunization Coverage Good (If any two points mentioned) Average (if any one point mentioned) No Response (if no points mentioned) Give overall assessment of the trainers based on responses received above, by giving scores out of 5 Count all the ratings given for the above 5 questions 5 Points: If at least 3 excellent OR 5 good 4 points: If at least 2 excellent OR 4 good 5 4 (excellent, good and 3 points: If at least 1 excellent OR 3 good 3 average) and then give 2 points: If no excellent point, but 2 good 2 scores out of 5 as detailed. 1 point: If 4 or more responses are average 1 D. Information regarding problems faced and suggestions: 1 Problems faced in Immunization Training (Ask open-ended questions) 2 Suggestions for future training courses related to: Course Contents Training methodology Class rooms and transport facilities at the training center No. of Trainers Flow of Funds as per Training norms Any other support required 3 Suggestions for changes in the Immunization Handbook: Areas to be added Areas to be deleted Areas to be modified E. Observe the facilities at the training centre (Tick appropriate response only if available at the training center by visiting the venue of training) 59

60 Classroom Black /white board Flip charts/marker pens LCD/VCD player Hostel-rooms Mess Water facilities Electricity Transport (own/hiring) No training facility available F. Major observations and suggestions of the Study Team:(Write the major gaps identified and observed ) Select two Blocks randomly (with trained health workers available) - one within15 kms and other more than 15 kms from district HQ; Collect the list of health workers trained from the selected blocks and the MPRs for last 3 months from the selected blocks. Tool to collect information from Block /PHC Medical officer A. General Information 1 State (name ): 2 District (name ): 3 Block/PHC (Name of Block or PHC being assessed): 3 Date of interview (date on which this format is filled) 4 Name/s of Investigators Name and designation of the 1. Medical officer 2. Study Tool No. 2 B. Routine Immunization Training status of health workers in the Block/PHC Category Staff in position Staff Received (number) training (number) ANM(HW-F) LHV (HA-F) HW (M) HA (M) Any other category (specify) Reasons if all not trained C. Routine Immunization services at the PHC (Talk with the medical officer and observe. Tick the appropriate 60

61 response) 1. Is RI Micro-plan with the following components available at the PHC? (Ask to see the micro plan of the block and observe the following) a) Map of catchment area including all sub-centers and distances from Yes No vaccine storage point b) Estimation of beneficiaries and logistics for current year Yes No c) Roster of health workers Yes No d) Alternate vaccine delivery plan to supply the vaccines and logistics to Yes No session sites? If not, mention the reason (by asking the interviewee) 2. Is Coverage monitoring chart/drop out chart displayed at the PHC Yes No 3. Any stock-outs or shortage (vaccines, syringes etc.) reported in last 3 Yes No months? (check records vaccine stock register) If yes, specify 4. How the disinfection and disposal of used syringes and needles is carried out? (Ask the medical officer and look for the waste disposal pit) 5. Mention the methods used for disposal of disinfected sharps (cut needles, broken vials & ampoules)? (Observe for any used syringes / vaccine vials lying scattered in PHC area) Disposal pit Other means specify No proper method 6. How many supervisory visits were undertaken to SC/Session site during last 3 months (write total number): By Medical Officer By other supervisors of HWs 7. Which records are available at the PHC to support the supervisory visits movement registers log book supervisory checklists/reports No records 8. Mention 1-2 areas in which you have noted change in the performance of HWs after training (Ask open-ended question initially. If unable to get the response, then prompt for areas such as micro planning, injection technique, recording and reporting, tracking of drop-outs, waste disposal, community mobilization etc. Note responses) D. Availability of equipment and supplies at the Block/PHC (Ask medical officer / cold chain handler / data handler and try to observe) Equipment and Supplies 1 Deep Freezers 2 Ice-lined Refrigerators (ILRs) 3 Voltage Stabilizers 4 Cold Boxes 5 Vaccine Carriers Number available Numbers functional Remarks if not functional 61

62 6 Icepacks 7 Thermometers Tick only if the following is available at the block Vaccine and Logistics indent forms Supply vouchers Issue register / record ADS 0.1 ml ADS 0.5 ml Disposable syringes Hub cutters Bleaching solution Waste Disposal pit Immunization cards Tracking bags Immunization register E. Cold chain and logistics support to the health workers at the PHC 1. Are temperature log books maintained correctly (temperature recorded twice daily; signatures of cold chain handler daily and MO weekly) 2. Is stock register maintained correctly (1. Check for entries, 2. Check stock entry of any 1 vaccine and cross-check with physical stock) 3. Is VVM in usable stages (Inner square is lighter than the outer circle) Refer to figure at end of Tool 2 4. Are ice-packs conditioned correctly (Ask cold chain handler to demonstrate) 5. Additional Comments: Yes Yes Yes Yes No No No No F. Major observations and suggestions of the study team: Vaccine Vial Monitors showing different stages Inner square lighter than the outer circle If the expiry date has not been passed USE the vaccine At a later time, inner square still lighter than the outer circle If the expiry date has not been passed USE the vaccine Discard point: Inner square matches colour of the outer circle DO NOT use the vaccine Inform your supervisor Beyond Discard point: Inner square darker than outer circle DO NOT use the vaccine 62