AN ASSESSMENT OF CURRENT RECORD SYSTEMS TO TRACK EARLY WARNING INDICATORS OF PEDIATRIC ARV RESISTANCE IN KENYA

Transcription

1 AN ASSESSMENT OF CURRENT RECORD SYSTEMS TO TRACK EARLY WARNING INDICATORS OF PEDIATRIC ARV RESISTANCE IN KENYA 30 September 2016 This publication was produced for the United States Agency for International Development. It was prepared by Jerry Okal, James Matheka, Irene Mukui, Jacob Odhiambo, and Nanlesta Pilgrim.

2

3 AN ASSESSMENT OF CURRENT RECORD SYSTEMS TO TRACK EARLY WARNING INDICATORS OF PEDIATRIC ARV RESISTANCE IN KENYA JERRY OKAL, JAMES MATHEKA, IRENE MUKUI, JACOB ODHIAMBO, AND NANLESTA PILGRIM

4 ACKNOWLEDGMENTS We are grateful to the care providers and staff in the 23 participating clinics as without their cooperation, the study could not have been implemented. We also thank the Kenyan Ministry of Health, especially the National AIDS and STI Control Programme, for their guidance and support throughout the support. We are also grateful for Palladium Kenya, especially Mr. Mwenda Gitonga and Mr. Joshua Oiro, for working tirelessly to update the electronic medical system with early warning indicators. We thank Waimar Tun, Sherry Hutchinson, Samuel Kalibala, and Scott Geibel for their support and commitment throughout the study and in finalizing this report. This report was made possible through support provided by the President s Emergency Plan for AIDS Relief and the U.S. Agency for International Development (USAID) via HIVCore, a Task Order funded by USAID under the Project SEARCH indefinite quantity contract (Contract No. AID-OAA-TO ). Published in September The Population Council Inc. HIVCore improves the efficiency, effectiveness, scale, and quality of HIV treatment, care, and support, and prevention of mother-to-child transmission (PMTCT) programs. The Task Order is led by the Population Council in partnership with Elizabeth Glaser Pediatric AIDS Foundation, Palladium, and the University of Washington. Suggested citation: Okal, Jerry, James Matheka, Irene Mukui, Jacob Odhiambo, and Nanlesta Pilgrim An assessment of current record systems to track early warning indicators of pediatric ARV resistance in Kenya, HIVCore Final Report. Washington, DC: USAID Project Search: HIVCore.

5 TABLE OF CONTENTS ACRONYMS...IV EXECUTIVE SUMMARY...1 INTRODUCTION...8 Background... 8 Early warning indicators... 8 The Kenya context... 9 Objectives...10 Methodology...10 Setting and participants...10 The interventions...11 Data collection...12 Measurement...13 Statistical analysis...15 Ethical considerations...15 KEY FINDINGS...17 Baseline findings...17 Intervention 1 findings: Quality improvement...22 Intervention 2 findings: Integration of early warning indicators into emr systems...30 DISCUSSION...32 Limitations...33 RECOMMENDATIONS...34 REFERENCES...36 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya iii

6 ACRONYMS AIDS ART ARV CDC EMR EWI HIS HIV HIVDR KAIS KII MoH NACOSTI NASCOP NGO PI PMTCT QI RA SOP UN USAID WHO Acquired immunodeficiency syndrome Antiretroviral therapy Antiretroviral Centers for Disease Control and Prevention Electronic monitoring record Early warning indicator Health information system Human immunodeficiency virus HIV drug resistance Kenya Aids Indicator Survey Key informant interview Ministry of Health National Commission for Science, Technology and Innovation National HIV/AIDS and STI Control Programme Non-government organization Principal investigator Prevention of mother-to-child transmission (of HIV) Quality improvement Research assistant Standard operating procedure United Nations United States Agency for International Development World Health Organization iv An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

7 EXECUTIVE SUMMARY Pediatric HIV patients are at greater risk for virological failure of antiretroviral therapy (ART) regimens and HIV drug resistance (HIVDR) than adults. In 2004, the World Health Organization (WHO), in collaboration with other agencies, recommended the surveillance of eight early warning indicators to combat HIVDR. Early warning indicators (EWIs) are quality of care indicators that specifically assess ART clinic programmatic factors associated with HIVDR without requiring laboratory testing for drug resistance. In 2011, an advisory panel refined the list of EWIs to five indicators: 1) on-time medication pick up, 2) retention in care, 3) pharmacy stock-outs, 4) dispensing of mono-or dual-therapy; and 5) viral load suppression. The fifth indicator is considered conditional, to be collected only at clinics with routine viral load monitoring. Regular EWI assessments are important in preventing HIVDR. In Kenya, the estimated number of children ages 0 to 14 infected with HIV in 2013 was 190,131 (10 percent of total HIV infections), with 138,000 in need of ART. A total of 60,141 pediatric patients were on ART (43.3 percent coverage). Studies among the pediatric population in Kenya reported that one third of children on ART experience virologic failure, and frequent (68 percent to 88 percent) drugresistant mutations were found among those who experience failure. Furthermore, retention of children on ART in Kenya is low. In 2012, the National HIV/AIDS and STI Control Programme (NASCOP) and the United States Centers for Disease Control and Prevention (CDC) in Kenya assessed the use of the WHO EWIs in 50 ART (including 32 pediatric) sites, revealing that routine monitoring of EWIs is not generally practiced. In response, NASCOP and CDC proposed to introduce the WHO EWI and build capacity to use the tool in sites across Kenya, including the same 32 pediatric sites from To date, this has not occurred, nonetheless research is needed to assess challenges of the tool s routine local use, to inform its introduction to sites across Kenya. OBJECTIVES The three objectives of the study were to: 1. Review current adherence assessment and EWI monitoring in Kenya in selected pediatric ART sites, and assess challenges and barriers to optimal monitoring. 2. Examine how facility decision-makers use EWI monitoring data to strengthen ART care for pediatric patients. 3. Develop and test specific EWI monitoring improvement strategies for strengthening pediatric ART program performance, for higher EWI target performance. METHODS The study was conducted from May 2015 to June 2016 in 23 of the original 32 pediatric ART sites in Kenya. A one-group pre-post intervention study design assessed current EWI monitoring, identified challenges and barriers to optimal monitoring, and identified strategies to improve monitoring. An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 1

8 A formative assessment was conducted in all the facilities to 1) inform the design of the intervention and 2) to provide baseline data for the intervention s evaluation. An endline assessment was conducted three months after the intervention. Data were collected at both assessments using a facility assessment survey, provider survey, key informant interviews (KIIs), and chart abstractions. Although not within the original study plan, at endline EWIs were calculated at the intervention sites at the request of NASCOP. The first four EWIs were calculated using the WHO EWI spreadsheet. EWI 5, virological suppression, could not be calculated, because it requires viral load information 12 months after initiation of ART during the extraction period. There were patients who initiated ART during the extraction period but had not been on ART for the 12 months required for viral load to be calculated. For other patients, the viral load information at 12 months was missing from their records. Intervention Selection Process The EWI improvement strategies were determined by the formative assessment, in consultation with NASCOP, and during a key stakeholder meeting. Of priority were improvement strategies easily replicated with constrained resources, and strategies directly addressing the challenges associated with properly documenting service statistics necessary for EWI calculation. Two interventions emerged focused on improving patient record-keeping: 1) a quality improvement approach whereby facilities identified and implemented procedures that improved their EWI monitoring abilities and 2) integrating EWIs into EMR systems. Intervention Facility Selection Of the 23 facilities, 10 facilities were purposively chosen to receive the intervention based on three criteria. First, facilities with large amounts of information missing from medical records precluding EWI calculation were considered for inclusion. Second, facilities were chosen for representation of the types of record-keeping systems in Kenya electronic medical records (EMRs) and paper-based records. Third, facilities with EMR systems managed by Palladium, a partner in the study, were included. Intervention 1: Quality Improvement The first intervention focused on quality improvement using the small test of change (STOC) approach, which encourages facilities and utilizes quality improvement (QI) teams to: 1) examine their current monitoring procedures; 2) determine what improvements are needed; 3) set goals; and 4) devise tests of change that may facilitate achievement of their goals. Instead of focusing on sweeping institutional changes, the STOC approach challenges QI teams to identify the single greatest change that can be made within one week that could result in improvement, and QI teams try the change on a small scale and track indicators to see if the test results in any improvement. If there is improvement, QI teams try the change on a larger scale. If a change does not result in improvement, the QI team assesses why, improves the strategy, or tries a new one. The discrete steps of the STOC approach are: 1) creating a QI team, 2) determining a goal, 3) choosing a strategy that can be implemented in one week, 4) establishing measures, 5) testing the chosen strategy, 6) evaluating the performance, 7) planning and implementing changes on a larger scale, and 8) monitoring performance over time. 2 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

9 Intervention 2: Integration of EWI into EMR systems The second intervention focused on enabling EMR facilities generation of EWIs. In the EMR facilities, Palladium updated the system with data fields to capture information for calculation of each EWI not already in the system from standard Kenyan health forms. The update also allowed health staff queries and a specific EWI calculation for a facility. Palladium provided necessary technical support to these facilities following the training. Intervention Implementation A 10-hour training on the STOC approach and EMR system update was conducted in each of the 10 facilities. The training also provided information on EWIs, how they are measured, and how EWIs can reduce HIVDR. Facilities were only able to commit 10 hours of training due to their clinical operations; some facilities chose a one-day training, while others chose training over two or three days. Training at facilities allowed for all levels of staff involved in HIV care at a facility to be trained, as off-site training would have resulted in only one or two staff from each facility being trained. On-site training also allowed all staff to participate in planning the facility s quality improvement strategy, providing a sense of ownership for a chosen strategy. Data Collection and Analysis A total of 23 facility administrators completed the facility assessment and participated in the KIIs at baseline and endline, and 32 and 24 providers completed the provider survey at baseline and endline, respectively. A total of 1,923 and 1,783 charts were abstracted at baseline and endline, respectively. Descriptive statistics were generated for quantitative data, and thematic content analysis was conducted for qualitative data. Data triangulation of the qualitative and quantitative data allowed a comprehensive understanding of facility readiness for EWI monitoring as well as the impact of the pilot intervention. The patterns and themes identified from the qualitative data were used to contextualize and, at times, explain the results of the quantitative data. FINDINGS Intervention Strategy 1: Quality Improvement Readiness to monitor EWI 1: On-time medication pick-up With the exception of two indicators, the medical and pharmacy records contained for calculation of ontime medication pick-up (EWI 1); daily dose of medication dispensed and amount of medication picked up were missing from medical records at baseline. Challenges associated with EWI 1 identified at baseline included lack of space on documents for recording necessary information and lack of standardized procedures at facilities to support pill pick-up. As a result of the intervention, the intervention facilities designed their own forms to document information and sent patients reminders to pick up their medications. At endline, chart abstractions revealed more facilities with the required information for calculating EWI than at baseline. An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 3

10 Readiness to monitor EWI 2: Retention in care At baseline, administrators noted a number of challenges that made tracing patients and retaining them in care difficult (EWI 2), including incorrect contact information from patients, orphaned patients switching caregivers, and caregivers not bringing their children to appointments. As a result of the intervention, intervention facilities utilized cell phones to call patients or caregivers who missed appointments, asked caregivers for multiple contact numbers, and instituted a policy encouraging caregivers to bring their children to appointments. Despite these strategies, the number of facilities who traced patients who missed their appointments remained unchanged at endline. Several intervention sites cited high staff turnover and shortages in the months following the intervention in preventing full implementation of strategies. Readiness to monitor EWI 3: Pharmacy stock-out There was a slight increase in facility administrators and providers reporting that ARV stock-outs (EWI 3) was a problem at their facilities from baseline to endline. These results should be interpreted with caution as there was no timeframe associated with the question. At endline, facility administrators at intervention sites reported being proactive to ensure stock-outs do not occur by keeping track of medication quantity and ordering medications when supply became low. Readiness to monitor EWI 4: Dispensing practices In endline qualitative interviews, administrators from intervention sites reported having problems with the pharmacy dispensing dual therapy regimens (EWI 4). In response, they required that patients return to the providers to check that they had received the correct regimen and encouraged patients/guardians to know the names of the medications so they too can assess whether they received the correct regimen. Readiness to monitor EWI 5: Viral load suppression In comparison to baseline, more intervention sites at endline had viral load information in their records. The main challenge that was identified at baseline to EWI 5 was the long time it took for facilities without viral load machines to receive viral load results. Despite this challenge, facilities were working to ensure that all patients had a viral load test conducted and the results documented in their records. Early warning indicators calculation Early warning indicator (EWI) EWI 1 On-time medication pick-up EWI 2 Retention in care EWI 3 Pharmacy stock-outs EWI 4 Dispensing practices EWI 5 Viral load suppression (where viral load testing is available) WHO recommended target More than 90 percent of pick up is on time More than 85 percent retained after 12 months of ART 100 percent of 12 month period without stock-outs 0 percent dispensing of mono- or dual-therapy (100 percent triple therapy dispensed) More than 85 percent viral load suppression after 12 months on ART The calculation of the EWIs using the WHO spreadsheet showed that intervention facilities met the target of 100 percent of a 12-month period without drug stock-outs (EWI 3). Seven of the facilities met the target for 0 percent dispensing of mono-or-dual therapy (EWI 4); in three facilities, EWI 4 could not be calculated due to missing information. Only three of the ten facilities met the target of more than 4 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

11 90 percent on-time medication pick-up (EWI 1). One facility met the target of more than 85 percent of patients retained in care after 12 months of ART (EWI 2). Intervention 2: Integration of EWIs into EMR Systems Three of the five EWIs on-time medication pickup, retention in care, and prescribing practices were successfully integrated into the IQCare platform, the EMR system managed by Palladium. The EMR system was also able to calculate all three indicators. Facilities chose not to include dispensing practices due to system malfunctions and viral load suppression could not be programmed. There were challenges with the integration, including system breakdowns, interconnectivity issues between computers, and power outages. Staffing issues were a serious issue for facilities and directly influenced their ability to effectively monitor EWIs and continuously use the updated EMR system. Facilities reported being understaffed or experiencing high staff turnover or transfers; the former resulted in little time to document care in records as staff time is dedicated to patient care, and the latter resulted in newer staff not having the knowledge about EWIs. RECOMMENDATIONS WHO developed EWIs to aid ART facilities in identifying factors associated with HIVDR without relying on viral load results an innovative solution for resource constrained settings. Through the routine abstraction of data from clinical records using the WHO EWI Excel tool, clinics can use the results to identify problems that should be addressed to minimize HIVDR. However, the usefulness of EWIs is highly dependent on the quality of record-keeping systems. It is negatively affected by weaknesses in the current record-keeping system, including incomplete records and missing data. Further, systemic issues (e.g. high staff turnover, staff shortages, and long turnaround times for viral load results) affect facilities abilities to monitor EWIs and implement strategies for improving EWI performance. A twopronged approach, based on results from this study, is recommended for EWIs to be integrated into routine clinical monitoring to improve patient care. The first is centered on implementing strategies to calculate EWIs immediately and providing facilities with their results. The second prong is focused on implementing strategies to build facilities capacity to routinely collect the data. These strategies must be practical, executable and sustainable within the country context. The following recommendations expand upon these two approaches. Recommendations to facilitate the immediate calculation of EWIs The baseline results of the study showed that a large proportion of pediatric ART facilities are not ready to routinize EWI monitoring. Lack of knowledge about EWIs, weaknesses in the current record-keeping system, and systemic issues are significant impediments at the facility level for monitoring of EWIs. Nonetheless, the calculation of EWIs can be facilitated by: 1. Assigning individual(s) at the county level to periodically collect EWIs. A county EWI specialist should be designated to visit ART facilities on a regular basis to abstract clinical record information using the WHO EWI tool and calculate EWIs. Each clinic should receive its results and address the barriers hindering optimal performance. An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 5

12 2. Calculating and reporting the completeness of the data from which EWIs were calculated. A facility where an EWI is calculated based on 90 percent complete data requires a different approach than a facility where an EWI is calculated on 40 percent complete data, for example. Decision makers at the national, county and facility level should have the full context of data quality to inform the strategies they implement. 3. Continuing to integrate EWIs into the EMR platforms. Using the WHO EWI Excel spreadsheet requires a great deal of time and human resources to abstract charts. The integration of EWIs into the EMR platform alleviates these issues and allows for quick calculation of EWIs because chart abstraction will not be needed (at least for EMR-enabled facilities). Palladium was successful in integrating three EWIs into its EMR platform. With time, they will address the challenges encountered and all EWIs will be integrated in the IQCare platform. Therefore, a roadmap exists on how to integrate EWIs into different EMR platforms across Kenya. Further, this integration can be completed during routine system updates and upgrades. Recommendations to foster the routine facility-level monitoring of EWIs Routine facility EWI monitoring is critical for identifying patients at risk of HIVDR in a timely manner. For example, regular review of patient appointment keeping and drug pickups can quickly identify those patients at risk of HIVDR. The baseline results of the study indicated the need to build capacity around improving the record-keeping systems and the use of EWIs. The endline results of the study show that a quality improvement strategy can mobilize facility staff to take practical steps to improve their recordkeeping systems, thereby facilitating their ability to monitor EWIs. Routine facility EWI monitoring can be fostered by: 4. Developing or building upon existing quality improvement platforms to improve record-keeping systems and facilitate EWI use. Baseline results show that clinical staff was generally unaware of EWIs and did not recognize the importance of good record-keeping systems. Therefore, quality improvement must focus on educating and training staff to improve record-keeping systems, monitor EWIs and use the results to improve patient care. To effectively do this, buy-in from staff is critical and creates a sense of ownership. An approach, such as the STOC used in this study, can be developed or integrated into existing quality improvement platforms because of its focus on involving all levels of the facility and on making small changes rather than sweeping ones. The creation of champions, through the development of quality improvement teams at each facility, can also facilitate this process. 5. Assigning an individual(s) at the county level for EWI quality improvement training and improving record-keeping systems, and monitoring progress at all ART facilities. Stakeholders in the current study were firm in their assessment that traditional training models where one or two staff members from each facility are convened for a large training and are expected to train their colleagues were inefficient and ineffective in a context of high staff turnover. The training must be given at each facility to ensure continuity of the quality improvement process due to larger number of staff possessing the knowledge of EWIs and procedures set in place to improve monitoring. A training specialist at the county level should conduct the training at each facility for all relevant staff, provide booster sessions as needed, and support the quality improvement teams. 6 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

13 6. Encouraging and allowing facilities to identify and lead the implementation of quality improvement strategies for EWI and improving record-keeping systems. Facilities differ in the challenges associated with their readiness and therefore, ability to effectively monitor EWIs. A quality improvement strategy that works for one facility might fail in another. The staff of the facilities are in the best position to identify where the gaps and problems exist in their facility that impede data completeness and EWI monitoring. They are also in the best position to develop strategies to address the gaps and problems with appropriate training and support. These recommendations must be implemented in an environment with concomitant systemic changes to improve monitoring and patient outcomes. CONCLUSION Pediatric ART sites were receptive to the intervention, identifying strategies to improve EWI monitoring and taking steps to improve their abilities for EWI monitoring. Systemic issues (e.g. high staff turnover, staff shortages, and long turnaround times for viral load results) must be addressed, however, for optimal performance. Some EWIs were successfully integrated into an EMR system. Furthermore, although the WHO EWI tool is an excellent indicator of factors associated with ART resistance, it is highly dependent upon medical record completeness, and continued quality improvement strategies must be implemented to ensure accurate information from EWI monitoring. An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 7

14 INTRODUCTION BACKGROUND Pediatric HIV patients are at greater risk for virological failure of antiretroviral therapy (ART) regimens and acquiring drug-resistant HIV than adults (Hamers et al. 2013). Children born to HIV-infected mothers are frequently exposed to ARV mono-or duo-therapy regimens 1 in utero unless the mothers are given trivalent therapy during pregnancy. As such, resistance to first-line ARV regimens may be present in up to 60 percent of perinatally-infected infants, for whom PMTCT failed (Hamers et al. 2013; Kityo et al. 2016). For example, a 2016 study from Uganda reported HIV drug resistance (HIVDR) was present in 10 percent of 319 children younger than 12 years of age, with HIVDR occurring in 35.7 percent of PMTCT-exposed children, 15.6 percent in children with unknown PMTCT history and 7.7 percent among children who have never used ARV (ARV-naïve) (Kityo et al. 2016). Another contributor to treatment failure and eventual development of HIVDR is poor adherence, which is prevalent during early childhood and among adolescents (El-Khatib et al. 2011; WHO 2012). For example, among perinatally infected adolescents with high viral loads ( 5000 copies/ml) development of resistance was observed for nucleoside reverse transcriptase inhibitors (62 percent), non-nucleoside reverse transcriptase inhibitors (57 percent), and protease inhibitors (38 percent) (Tassiopoulos et al. 2013). There is a need to strengthen health system to support ART adherence and retention in care to reduce HIVDR risk among pediatric patients. There is a lack of monitoring tools suitable for addressing treatment adherence among children and adolescents, however (Haberer and Mellins 2009, Hamers et al. 2013). For example, subjective measures, such as self-reporting, are often influenced by a caregiver and cannot measure adherence of infants and toddlers directly (Haberer and Mellins 2009). One globally implemented strategy for assessing and preventing pediatric HIVDR is the surveillance of early warning indicators (EWIs). EARLY WARNING INDICATORS The surveillance of EWIs is one of the most widely implemented components of WHO s strategy for assessing and preventing HIVDR (WHO 2012a). Developed in 2004 by WHO, in collaboration with other agencies, EWIs are quality of care indicators specifically assessing programmatic factors at individual ART clinics associated with emergence of HIVDR, without requiring laboratory testing for drug resistance. Initially, there were eight EWIs, but in 2011, an advisory panel reviewed and simplified definitions and refined the list to only five EWIs. Table 1 presents the five EWIs and associated targets recommended by WHO. The fifth EWI (viral suppression) is considered conditional and designed for implementation only at clinics with routine viral load monitoring. EWIs should be properly monitored and resulting information used to address sub-optimal ART program performance if their results will help reduce the risk of HIVDR emergence. Regular EWI assessments are important to: a) provide an evidence 1 Standard recommended HIV first-line regimens contain three drugs (two nucleoside reverse transcriptase inhibitors in combination with one non-nucleoside reverse transcriptase inhibitor). The prescribing of mono-or dual drug regimens, which consists of one or two drugs respectively, is known to be associated with drug resistant HIV. 8 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

15 base for programmatic change and public health action to prevent and address HIVDR and b) to assess the extent to which ART programs optimize HIVDR prevention. Table 1 Early warning indicators calculation Early warning indicator (EWI) EWI 1 On-time medication pick-up EWI 2 Retention in care EWI 3 Pharmacy stock-outs WHO recommended target More than 90 percent of pick up is on time More than 85 percent retained after 12 months of ART 100 percent of 12 month period without stock-outs EWI 4 Dispensing practices 0 percent dispensing of mono- or dual-therapy (100 percent triple therapy dispensed) EWI 5 Viral load suppression (where viral load testing is available) More than 85 percent viral load suppression after 12 months on ART THE KENYA CONTEXT At the end of 2013, an estimated 190,131 children ages 0 to 14 were infected with HIV in Kenya (10 percent of total HIV infections), with 138,000 in need of ART. A total of 60,141 pediatric patients were on ART (43.3 percent coverage). The number of women believed to be in need of PMTCT each year is approximately 80,000 (National AIDS/STI Control Program (NASCOP) 2014). There are no known estimates of virological failure and HIVDR among Kenya s pediatric population. Two studies, however, provide insights into the possible magnitude of virological failure and HIVDR in this population. In a 2013 study of 100 children, ages 18 months to 12 years, 34 percent experienced virological failure, and 68 percent of those with failure had drug-resistant mutations (Wamalwa et al. 2013). A 2014 study of 462 children younger than five years old in 15 sentinel sites reported 33 percent virological failure, but drug resistant mutation was higher (88 percent) (Ngugi et al. 2014). To reduce the risk of HIVDR, in 2012 NASCOP with support from Centers of Disease Control and Prevention (CDC) assessed the use of the WHO EWI in 32 pediatric ART sites, with good prescribing practice in 98 percent, moderate patient retention (72 percent of patients retained at 12 months), and poor appointment keeping (29 percent kept >80 percent of appointments) (Ngugi et al. 2013). Based on these results, NASCOP recognized the need to expand training and implementation of the WHO EWI tool for facilities across the country. To date, however, the EWI tool has not been introduced into facilities. This study aimed to inform NASCOP s training and implementation of the WHO EWI tool by assessing challenges and barriers for its routine local utilization, and testing strategies to increase the acceptance and utilization of the WHO EWI tool. The study was conducted under the HIVCore project, which is funded by the U. S. Agency for International Development (USAID) and led by the Population Council. The study team was comprised of researchers from NASCOP, Palladium, and Population Council. An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 9

16 OBJECTIVES The study had three objectives: 1. Review current adherence assessment and EWI monitoring, and assess challenges and barriers to optimal monitoring in selected pediatric ART sites in Kenya. 2. Examine how facility decision-makers use EWI monitoring data to strengthen pediatric patient ART care. 3. Develop and test specific EWI monitoring improvement strategies for strengthening pediatric ART program performance, for higher EWI target performance METHODOLOGY The study, one-group pre-post intervention (Shadish et al. 2002), was conducted from May 2015 to June 2016 in 23 of the original 32 pediatric ART sites in Kenya. SETTING AND PARTICIPANTS In 2012 the Kenyan National AIDS/STI Control Program (NASCOP) assessed EWI monitoring in 32 pediatric facilities selected by a stratified sampling to ensure ART clinic representation in Kenya, in all eight regions: Central, Coast, Easterm, Nairobi, North Eastern, Nyanza, Rift Valley, and Western (Ngugi 2014). Stratification was by facility type (e.g. health center, district hospital), size (small or large) and administration ( MoH or faith-based). With the exception of Nairobi, an entirely urban region, all other regions comprised rural and urban areas. The regions also varied in HIV rates: highest in Nyanza (15.1 percent) and lowest in North Eastern (2.1 percent) (NASCOP 2013). The current study included 23 pediatric ART health facilities representing a subset of the 32 facilities from the 2012 NASCOP study. Facilities were selected to be representative of geographic region and facility type. (The North Eastern province was not included in this study due to political unrest during the formative assessment.) Formative Assessment A formative assessment was conducted in the 23 facilities 1) to inform the design of the intervention and 2) generate baseline data for its evaluation. A facility assessment survey, provider survey, key informant interviews, and chart abstractions assessed current EWI monitoring, identified challenges and barriers to optimal monitoring, and identified strategies to improve EWI monitoring. The intervention strategies selected for this study were developed based on the results of the formative assessment. Endline Assessment The formative assessment instruments were again administered three months after the intervention, constituting the endline assessment. Intervention Facility Selection Of the 23 facilities, 10 were purposively chosen for the intervention (see description below) based on 10 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

17 three criteria. First, facilities with large amounts of information missing from medical records precluding EWI calculation were considered for inclusion; missing information ranged from 44 percent to 67 percent in these facilities, compared to better performing facilities where missing information ranged from 0 to 22 percent. Second, facilities were chosen to represent types of record-keeping system in Kenya electronic medical records (EMR) and paper records. Third, for facilities with EMR systems, those managed by Palladium, a partner in the study, were included, and those chosen had the IQCare platform deployed and operational, according to Palladium. Six facilities with EMR systems and four facilities with paperbased systems, in five regions Eastern, Coast, Rift Valley, Central, and Nairobi participated in the intervention. THE INTERVENTIONS The interventions developed for this study focused on improving EWI monitoring systems in a subsample of 10 facilities from the 23 baseline facilities. The goal of these interventions was to examine the feasibility of implementing EWI improvement strategies. This study tested whether a) pediatric ART facilities can implement the changes, from management and technical capacity perspectives, and b) how the new strategy improved facilities readiness for EWI monitoring. Intervention Selection Process The EWI improvement strategies were determined by the formative assessment, in consultation with NASCOP, and by a key stakeholder meeting with representatives from government institutions and clinical facilities. The formative assessment revealed lack of EWI awareness, poor record keeping, lack of training for effective monitoring of HIVDR indicators such as adherence monitoring, and lack of staff training on monitoring HIVDR at every level. Improvement strategies that can be easily replicated, given constrained resources, were of priority to NASCOP and key stakeholders. They were also interested in strategies directly addressing the challenges associated with proper documentation of service statistics for EWI calculation, ultimately leading to high quality program performance. Based on these results, the interventions (described below) emphasized improving patient record-keeping in two ways: 1) facility identification and implementation of standard operating procedures to improve EWI monitoring abilities and 2) integration of EWIs into EMR systems. Intervention 1: Small test of change (STOC) for quality improvement: EMR & paper-based facilities In the six EMR and four paper-based facilities, the intervention focused on quality improvement (QI) utilizing the STOC approach (Stafford and Amoroso 2007, Langley et al. 2009, HRSA 2011). The STOC approach encourages facilities, through the creation of QI teams, to 1) look at their current treatment outcomes and monitoring procedures, 2) determine what improvements are needed, 3) set goals for where they want to be, and 4) devise tests of change that may help them achieve their goals. Instead of focusing on sweeping institutional changes, the STOC approach challenges QI teams to identify the single greatest change that can be made in one week that could result in improvement. QI teams then implement the change on a small scale and track their indicators to see if their test resulted in any improvement. If there was improvement, the QI team introduces the change on a larger scale. If that change results in no improvement, the QI team assesses the factors and makes improvements to the An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 11

18 strategy or tries a new one. The steps of the STOC approach are: 1) create a QI team, 2) set a goal, 3) chose a strategy that can be implemented in one week, 4) establish measures, 5) test the strategy chosen, 6) evaluate performance, 7) plan and implement changes on a larger scale, and monitor performance over time. A 10-hour training on the STOC approach was conducted in each of the 10 facilities. Facilities were only able to commit to 10 hours to the training due to their clinical operations. Some facilities were trained in one day while others were trained for a few hours over two to three days. Training at facilities allowed for all levels of staff involved in HIV care (e.g. administrator, provider, pharmacist, lab technician) at a facility to be trained, as off-site training would have resulted in training of only one or two staff members. Formative assessment findings in addition to stakeholders noted that interventions that trained only one or two staff have had little success due to high staff turnover and transitions; training all staff helps ensure continuity. On-site training also allows all staff to participate in planning a facility s quality improvement strategy, providing a sense of ownership of the chosen strategy. The training provided information about EWIs, how they are measured, and how EWIs can be used to reduce HIVDR. The training guided facilities through the STOC approach and assisted each facility in using STOC to plan its best suited quality improvement strategy to improve its ability to monitor EWIs. Intervention 2: Integration of EWI indicators into EMR systems The second intervention focused on enabling EMR facilities generation of EWIs. In the EMR facilities where the IQCare platform was deployed, Palladium updated the system with data fields to capture the information needed to calculate each EWI not already in the system from standard Kenyan health forms. The update also allowed a query by health staff and calculation of the specific EWI for that facility. During the one-day training, participants were trained by Palladium on the new additions to the IQCare, including how to enter data and retrieve EWI indicator data. Palladium provided necessary technical support to the facilities following the training. EMR facilities were also trained on the QI processes to ensure data were entered into the EMR system. In some facilities insufficient computers did not allow the system to be used during patient visits, or unexpected power outages rendered the system useless, and as a result, patient visit information was recorded on paper forms and later entered into the EMR. If information is missing or not entered into the EMR, it reduces a facility s ability to monitor EWIs. DATA COLLECTION Data collected during the formative assessment served as the baseline for the study. Endline data were collected three months after the interventions. Baseline and Endline Data Collection Baseline and endline data were collected by a team of Kenyan research assistants (RAs) with clinical backgrounds supervised by the study s principal investigator and study manager. A three-day intensive training for baseline and endline data collection focused on EWIs, reviewing study data collection instruments, data collection protocols, interviewing skills, and ethics in human subjects research. 12 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

19 To facilitate data collection, NASCOP and the Ministry of Health (MoH) provided signed letters stating the study s purpose that requested the support of the Officer in Charge of the participating pediatric ART clinics. Appointments for data collection were made by the study manager with each facility. At each facility, RAs administered the data collection instruments to appropriate staff. Facility health staff assisted the RAs with chart abstractions of medical and pharmacy records for information to assess facilities readiness to monitor the five WHO EWIs. Training for Intervention Implementation The intervention was led by Kenyan clinical officers knowledgeable in HIV care and facility procedures. The study s trainers were trained by Population Council s principal investigators over three intensive -days, covering various topics including pediatric HIVDR in Kenya, EWIs and their measurement, effective facilitation, IQCare system use and updating, QI and the STOC approach, and ethics in human subjects research. Throughout the training, trainers were given ample opportunities to practice and receive suggestions from the principal investigators and study manager on improvement. At each facility, the training was led by two RAs. The study manager contacted each facility to schedule the best day for its facility training. After the training, RAs completed an evaluation form detailing what information was, and was not, covered, as well as how they perceived the training was received by participants. MEASUREMENT Facility Assessment Survey The facility assessment survey collected information on the quality of information in patient files that are used to measure EWIs. Trained RAs with clinical backgrounds administered the facility assessment survey to the person in each facility most knowledgeable of facility procedures, such as a facility administrator, Officer in Charge, or pharmacist. The survey s three sections elicited information on facility procedures and the state of its medical and pharmacy records. At baseline, survey questions referred to the prior 12 months. At endline, the survey questions referred to the last three months, the period of time that had elapsed since the intervention. Section 1 collected descriptive information about the facility, such as type of facility and administrator qualifications. Section 2 collected information on facility capacity and practices, measuring frequency of viral load testing, frequency of pill counts, patient adherence and tracing, and percentage of patient records recording when relevant procedures were conducted. Section 3 collected information on the completeness of medical and pharmacy records, including information needed for calculating EWIs such as percentage of medical records clearly stating date of ART initiation, dates of ART pick-up, and amount of medication dispensed. Key Informant Interviews Key informant interviews (KIIs) were conducted in English with a facility official who was fully knowledgeable of the pediatric ART program and procedures at baseline and endline, with 22 KIIs (two facilities had the same administrator) conducted, by trained RAs with clinical backgrounds and lasted one An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 13

20 hour. The KII guide comprised questions that generated discussion on facility pediatric ART treatment procedures; existing pediatric ART record-keeping, monitoring and evaluation systems; current facility procedures to assess facility and patient adherence; current compliance with specific record-keeping of key indicators necessary for WHO EWI monitoring; experience with the current EWI monitoring system; and identification of resources necessary for improved monitoring, and characteristics of the facility s desired optimal EWI system. At endline, KIIs focused on facility procedures, standards and policies for EWI monitoring. Rapid KIIs at intervention facilities, lasting 10 to 15 minutes, at one month and two months after the intervention, assessed what changes, if any, were implemented in a facility, staff reactions, and challenges. These follow-ups provided additional information for understanding the impact of the interventions. Provider Survey The provider survey assessed their roles, capacities for supporting EWI monitoring, and the pediatric ART staff performance at facilities. The survey comprised three sections. Section 1 collected demographic information on providers including age, sex, position at facility, and years at the facility. Section 2 collected information on providers perceptions of HIVDR monitoring at the facility, including current state of medical records, staff training and abilities for monitoring HIVDR, and prescribing practices. Section 3 collected information on providers knowledge of EWIs, EWI training and use, and confidence in using EWIs, if a provider had been trained or used EWIs. The survey was administered to pediatric ART staff, including treatment practitioners, pharmacists, monitoring and evaluation coordinators, or any staff identified as essential to pediatric ART care and monitoring. All pediatric ART staff present on the day of survey were recruited for the survey, resulting in a sample of 34 at baseline and 24 at endline. The survey was administered using handheld computers by trained RAs with clinical backgrounds. Chart Abstraction Chart abstraction of all available medical and pharmacy records of pediatric HIV patients in each facility assessed readiness for monitoring the five WHO EWIs. Readiness was defined by patient charts with information necessary for calculating EWIs. Using handheld computers, trained research assistants with clinical backgrounds used a chart abstraction tool to record whether specific information was present in each chart. From the medical records, the presence or absence of the following information was recorded: viral load test results and dates, ART initiation dates, pill count and adherence counseling conducted at last visit, and patient status if applicable (e.g. date of death, date ART stopped, date transferred facility). At the pharmacy, the presence or absence of the following information was recorded: dates of ARV pickup, regimen picked up, and number of days or daily dose of ARV picked up. A total of 1,923 and 1,783 charts were abstracted at baseline and endline, respectively. EWI calculation EWI calculation was not within the scope of the original study due to resource limitations, but after presentation of the baseline results, NASCOP requested EWI calculation. In response, EWIs were calculated at the intervention sites with resources available. At endline, EWIs 1 to 4 were calculated in the intervention facilities covering a 12 to 16 month period, from January 2015 to June 2016, using the 14 An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya

21 WHO EWI spreadsheet. EWI 5, virological suppression, could not be calculated, because it requires viral load information 12 months after initiation of ART during the extraction period. At the study sites, there were patients who initiated ART during the extraction period but had not been on ART for 12 months for viral load to be calculated. For other patients, the viral load information at 12 months were missing from the records. STATISTICAL ANALYSIS Quantitative Survey data and chart abstraction data were analyzed in Stata version 13. A facility with greater than 75 percent of records with necessary information was considered ready to calculate EWIs. Seventy-five percent was chosen as the threshold to align with the various data collection methods. For example, administrators and providers were asked categorical questions on the state of their facilities records (e.g. what percent of patient records has the dates when their viral load was tested? (less than 25 percent, percent, percent, or greater than 75 percent). Chart abstraction data were dichotomized to greater than 75 percent, and less or equal to 75 percent. Frequencies were generated to describe facility and provider characteristics and facility ability to measure EWIs. The limited time after the intervention precluded analysis of changes over time, as indicator variables at baseline and endline were not comparable; baseline indicators referenced the last year while endline indicators referenced the previous three months. Whether improvements compared to baseline occurred in the three months after the intervention were qualitatively assessed. The calculation of EWIs utilized the WHO Excel spreadsheet. Qualitative All KIIs were audio-recorded and transcribed verbatim for analysis. Themes were explored and identified by research staff. We used thematic content analysis, a research method for the subjective interpretation of the content of text data through the systematic classification process of identifying themes or patterns (Hsieh and Shannon 2005). Triangulation Qualitative and quantitative data were triangulated for a comprehensive understanding of facilities readiness for monitoring EWIs as well as the impact of the pilot intervention. The patterns and themes identified from the qualitative data were used to contextualize and, at times, explain the results of the quantitative data. ETHICAL CONSIDERATIONS This protocol was reviewed and approved by the Population Council Institutional Review Board (Protocol 668) and the Kenyatta National Hospital University of Nairobi Ethics and Research Committee (KNH- ERC/A/81). The approving ethics committees did not require individual patient consent given the nature of the intervention. To protect facility administrators, we did not collect any personal identifying information to ensure they could not be identified. Although basic demographic information was An assessment of current record systems to track EWIs of pediatric ARV resistance in Kenya 15