QC for the Future: Laboratory Issues POCT and POL concerns

QC for the Future: Laboratory Issues POCT and POL concerns Valerie L. Ng, PhD, MD (University of California San Francisco (UCSF) and San Francisco General Hospital San Francisco, CA) DOI: 10.1309/7986T3PG123QEPWE Feature Received 7.11.05 Revisions Received 8.5.05 Accepted 8.8.05 Why is quality important? Because laboratory tests directly influence patient care. The introduction of routine quality control (QC) in the clinical laboratory was a major advance in improving the accuracy and reliability of clinical laboratory testing. The 1992 enactment of the Clinical Laboratory Improvement Amendments (CLIA 88) further established a minimum threshold for all aspects of clinical laboratory testing. Errors occurring during the analytical phase of testing within clinical laboratories are now relatively rare, and often caught by failed QC prior to release of patient results. Values produced by clinical laboratories are now rarely questioned, and the vast majority of errors now attributed to clinical laboratories actually occur in the pre- or postanalytical phase. 1,2 Point-of-care testing (POCT) occurs today in 2 major settings ideally with direct oversight and management by a clinical laboratory, or on its own in a physician office laboratory (POL). The first setting is typically that of medical centers, where the central clinical laboratory performs laboratory testing and oversees POC performed in the inpatient units and outpatient clinics. For this setting, the quality of the POC test is the single most important feature in choosing a method or device because most medical centers are accredited by the Joint Commission for Accreditation of Healthcare Organizations (JCAHO). A central tenet of JCAHO is that a single standard of care be used for the treatment of patients with the same disorder(s), regardless of where the patient is treated. Applied to laboratory testing, this means that any test for the same analyte, regardless of where the test is performed, must yield the same answer. Although a seemingly simple concept, this oversight is, in fact, a major task. If POCT is under the oversight of the clinical laboratory, then the answer to who s in charge is clear the laboratory director. If multiple POC test programs exist within a single medical center under multiple licenses, it becomes exponentially more difficult to assure that the directors of all programs are in agreement on all aspects of POC testing, and that all point of care (POC) test programs are in fact delivering the same standard of care. The original POC tests were those classified in CLIA 88 as waived tests. These tests are defined in CLIA 88 as tests so simple to perform that it was extraordinarily rare to obtain an incorrect result, and if an incorrect result were obtained, would pose no harm to patients; or those tests approved for home use. The original waived tests were fecal occult blood (FOB), nonautomated dipstick or tablet urinalysis (U/A), hemoglobin by copper sulfate (non-automated), spun hematocrit, ovulation and urine pregnancy tests (by visual color comparison), erythrocyte At least 70% of laboratory results are used for direct patient care diagnosis, therapeutic, or management decisions. The widespread acceptance that laboratory values are accurate has been applied broadly to include all laboratory tests, including those performed as point of care. sedimentation rate (non-automated), and blood glucose using devices approved by the Food and Drug Administration (FDA). The FDA and the Centers of Disease Control and Prevention (CDC) have historically been responsible for determining whether a test device would meet the waived categorization definition. The decision to categorize a device as waived has evolved and is currently based on extensive risk analysis and stress-flex studies coupled with field studies to demonstrate that the device meets the requirements of waived as defined in CLIA 88. An explosion of new waived test approvals has occurred in the last decade. Many of these newly approved waived tests were those that had not historically been performed as POC (eg, fingerstick glucose, group A Streptococcus rapid antigen detection, hemoglobin A1C, rapid influenza A antigen detection, etc). Also, a number of tests categorized as moderate complexity were redesigned as POCTs [eg, arterial blood gas (ABG), cooximetry, prothombin time/international normalized ratio, (PT/INR), electrolyte panels + hemoglobin/hematocrit + blood gases, etc]. Point of Care Versus the Laboratory The drive to develop POCT was related to the shift of patient care away from inpatient stays to outpatient arenas linked to the desire to provide ready access to laboratory testing regardless of rural versus urban settings. This shift in patient care often removed the patient from the proximity of the typical hospitalbased clinical laboratory, and exacerbated issues related to specimen transport and results communication. Thus, POCT was attractive for clinical care because it eliminated these issues and allowed real time therapeutic decision making. POCT occurs today with either the test device being brought to the patient or a specimen brought to a small laboratory equipped with many POCT devices (eg, POL). The widespread acceptance that laboratory values are accurate has been applied broadly to include all laboratory tests, including those performed as POCT. This is an unfortunate situation, in that POCT is not always performed under the same minimal conditions (eg, minimal test personnel requirements, quality control and assurance, validation, and verification) as conventional clinical laboratory testing. At a minimum, nonlaboratory professionals are typically the individuals performing POCT [eg, registered nurses (RNs); licensed vocational nurses, labmedicine.com October 2005 Volume 36 Number 10 LABMEDICINE 621

(LVNs); unlicensed personnel such as medical exam assistants, (MEAs); or others]. The vast majority of these non-laboratory professionals does not have basic laboratory quality training and lacks an inherent sense of quality testing innate to most laboratory professionals. The agency accrediting the test site also plays a role in the quality of test results. For example, a CLIA survey conducted by the Centers for Medicare & Medicaid Services (CMS) would require that the test site, test performance and test personnel adhere to the minimum requirements mandated by the manufacturer and CLIA 88; in contrast, a program accredited by either the JCAHO or the College of American Pathologists (CAP) would have additional POCT requirements, instituted to assure high quality of test results, to meet accreditation standards. The actual volume of POCT is relatively unknown. Outside of a few select automated POC instruments with connectivity and information system features, quantification of the actual number of manually performed POC tests can only be done manually. In 1996 and for waived tests, the National Inventory of Clinical Laboratory Testing Services estimated that 500 million waived tests were performed of which half were used to measure blood glucose levels, representing 7% of all laboratory tests performed. 3 The Issue of Quality Control Point of care testing broadly includes any laboratory test of any complexity (waived, moderate, high) typically performed near the patient and usually outside of a clinical laboratory. There is extremely limited published literature on the quality of POC tests and their impact on patient care. The limited published literature in this area addresses waived POCT only, of which 2 are noteworthy. The first study demonstrated that 46% of waived tests were used for a definitive diagnosis (with no additional confirmatory testing), 42% for screening purposes only, and 9% to monitor previously diagnosed patients. 4 Quality control was run in 56% of the 920 laboratories surveyed, with no distinctions noted for POLs, hospital/independent laboratories, CLIA- and state-regulated laboratories, or accredited and nonaccredited laboratories. External liquid controls were run at a higher frequency if testing was performed for a definitive diagnosis (67%) or to monitor patients already diagnosed (60%) than for screening (47%). Of note, sites employing clinical laboratory scientists (CLSs) had a higher rate (59%) of liquid external control use than sites that did not employ CLSs (39%). There was tremendous variation in quality control perceptions and practices. Given this variability and lack of consistent use of quality control, the authors recommended that built-in controls be included in every unitized device, so that optimal testing practice would occur with each test. As stated in this study, Each patient test should be a quality control event. Similar findings were obtained when the study was extended to 3 states (Arkansas, New York, Washington). 3 This study was conducted as part of an ongoing monitoring program established by the CDC under CLIA, to assess the impact on regulation on laboratory testing practices. This study duplicated the findings of the original specifically, only 55% to 60% of laboratories followed manufacturer s QC requirements, with similarly low and more highly variable rates of compliance for other aspects of good laboratory practice. A higher level of compliance was observed for those laboratories accredited by external organizations. The conclusion of this study was that regulation alone was not sufficient to engender quality laboratory testing. Recent studies conducted by CMS of Certificate of Waiver (COW) laboratories have yet again reproduced these findings of less than acceptable adherence to good laboratory practice (http://www.phppo.cdc.gov/cliac/pdf/cliac0904.pdf; http://www.phppo.cdc.gov/cliac/pdf/addenda/cliac0204/ Addendum%20C_Yost_Certificate%20of%20Waiver_ February%202004.pdf; accessed March 16, 2005). A widely stated benefit of POCT has been that laboratory values discrepant with the patient s clinical condition can be readily detected. This is certainly a level of quality control not available to the typical clinical laboratory. However, this benefit may be overstated. One study from the laboratory s perspective determined an average rate of 0.8% in which a laboratory result (any level of complexity, including waived) did not correlate with clinical information. 4 A separate study from the clinician s perspective demonstrated that physicians were able to accurately predict a laboratory result only 57% of the time that was consistent with the clinical condition of a hospitalized patient. 5 Test Failures of POCT It is important to highlight some of the issues related to this backdrop of less than ideal performance of POC tests. A search of the published literature quantifying these issues was unsuccessful. I have thus resorted to recounting my personal experience with POCT. My experience is based on a setting where my hospital-based clinical laboratory is responsible for all POCT occurring on the campus including the acute care inpatient wards, ICUs, ORs, emergency departments, and 40+ outpatient clinics or treatment areas. My overall impression of POCT is that the average nonlaboratory professional user, despite the best possible design and fail-safe engineering of the test, will uncover an unanticipated way to make the test fail. Anecdotes from my personal practice supporting this impression include: Fecal occult blood. The manufacturer of our current POC test card states that the test pad must be read 2 minutes after applying the developer. The previous version had a test that could be read immediately after applying the developer. We had to switch from the previous immediate read version to the current 2-minute read version because the manufacturer of the immediate read version did not meet the requirements necessary to do business with the city and county of San Francisco (eg, equal benefits to domestic partners, no use of rainforest materials, etc). The physicians have now confused the 2 systems not only do they use the developer from the old system (that has been sequestered somewhere not obvious to the laboratory), but the test is read immediately after the wrong developer has been applied. Urine pregnancy. An outlying clinic originally obtained a negative result. After the patient had left, the test device was again examined and noted to have a positive result. (There was no information from the clinic as to the accepted time period during which the test result remained valid relative to the second test interpretation.) The patient had already been prescribed a drug not to be used in pregnancy and had been sent for a radiological procedure. A serum specimen obtained the same day and sent to the clinical laboratory confirmed a serum level of β-hcg level consistent with 1 to 2 weeks gestation. 622 LABMEDICINE Volume 36 Number 10 October 2005 labmedicine.com

An outlying clinic performed a urine pregnancy test. The first test yielded a negative result, not consistent with the patient s history of a positive home pregnancy test and clinical symptoms of nausea and vomiting. A second test with a second specimen was performed and yielded a positive result. The urine pregnancy test was repeated on both the first and second specimens; both specimens yielded positive results. Rapid group A Strep antigen testing. The particular test system used required the application of equal volumes of reagents A and B. Yet, a new vial of only 1 of the 2 reagents is often requested. 6 Reagent bottles containing clear solutions have been refilled with water. ( Looks like water, must be water. ) Reagents from different kits/lots are used interchangeably in a well-intentioned attempt to save money. Fingerstick glucose. A common issue is the ward returning a meter for service because it is broken. At least 90% of the time, the clinical laboratory determines that the meter is working perfectly fine. A patient s result is unexpected. Instead of following the recommended procedure to verify the instrument is working properly (ie, rerun, quality control materials), a supervising nurse quickly performs a fingerstick glucose test on herself to show the other nurse that the instrument is performing correctly. Hemoglobin. Three different levels of quality control materials are available low, normal, and high. Mothers and babies used to be housed separately. Point of care hemoglobin determinations in the separate units typically used 2 different levels of controls low + normal for mothers (because they are typically anemic), and normal + high for newborns (because they are typically plethoric). A recent change was implemented to house mothers and newborns together. There was confusion initially on why this unit now had to run all 3 levels of controls. Variable hemoglobin results were being obtained with a POC test instrument. When contacted, the manufacturer s customer service initially responded that the problem must be specimen or cartridge mishandling. A technical person was finally contacted who explained that the conductivity method used to measure hemoglobin would be affected by changes in protein concentrations in the patient s blood. The technical person further commented that hemoglobin/hematocrit (hgb/hct) generated values from this instrument should not be used for transfusion decisions. Not only is this information not in the package insert, it directly conflicts with its widespread use by anesthesia in the operating rooms for this exact decision making purpose. Finally, the technical person commented that the hgb/hct analyte has had more complaints to the manufacturer than for any other analyte, and acknowledged that there has been no official notification of users of this issue. 7 Test personnel were observed on their use of the POC test device. One person had blood smeared all over the cartridge (creating a biohazard risk), one inserted the cartridge upside down, and a third was unable to obtain a reading because dried up blood obscured the photometer lens. 8 Prothrombin Time/International Normalized Ratio (PT/INR). An instrument appeared in our operating room. Anesthesiologists were using the results to determine if a coagulopathy existed and the need for fresh frozen plasma infusion. The anesthesiologists had arbitrarily adopted the clinical laboratory s PT reference range, assuming (and without verifying) that the range was applicable to their POCT device. Furthermore, the anesthesiologists were unaware that PTs are widely variable depending on reagent/instrument, and that the INR was reproducible between laboratories only for patients who were on stable chronic anticoagulation treatment. The anesthesiologists were also under the impression that a device marketed for home use could be used reliably anywhere. This represents basic misunderstanding of 1) coagulation testing in general, and 2) devices approved for home use. Electronic QC. Some manufacturer s claim that electronic QC is sufficient to assure that the system works as expected. It must be acknowledged, however, that electronic QC exists in many versions some which test only the electronic guts of the instrument, and do not test issues related to specimen type or application. Other electronic QC may include a liquid that simulates specimen application but may consist of a matrix different than blood. As such, electronic QC with or without liquid QC material cannot be a replacement for regular QC involving patient specimen comparison with a reference test method since it cannot identify problems with test results related to substances in patients blood (ie, matrix effect). 9 Annual assessment of competency. Many sites find this to be an excessive burden, having to demonstrate the competency of each test person annually. The clinical laboratory has been asked to change the rules to either reduce the frequency of competency, or ideally eliminate the requirement of regular competency assessment altogether. Proficiency testing. This is the one area about which everyone complains the most. The sequence of events is: The proficiency test material is received with a limited period of time to have the POCT user perform testing and return results. The clinical laboratory person has to coordinate with the testing site to have someone available to perform the proficiency test. Despite setting up appointments with each test site, there is still wasted time while the Clinical Laboratory person tries to find POC test site user at the scheduled appointment time. Proficiency test material arriving in a glass sealed vial (eg, POC ABGs) creates yet another level of problems. The average POC tester does not know how to safely break open a sealed glass vial. The proficiency test material must be drawn up in a pipette and applied to the test device. The average POC tester is not proficient in pipetting, and often introduces excessive bubbles while drawing up the specimen, thereby changing at a minimum the po 2. If the proficiency test result is unsuccessful, an inordinate amount of time is spent identifying the cause of the error, developing and implementing a corrective action plan including retraining all relevant POC test personnel. Overall good laboratory practice. Our emergency department had been requesting POCT privileges for quite awhile specifically fingerstick glucose, urine dipstick, and urine pregnancy. After many meetings, labmedicine.com October 2005 Volume 36 Number 10 LABMEDICINE 623

the clinical laboratory agreed and explained that each test must have the following 4 items documented: Who did the test? Who was tested? Where is the result? What action was taken based on the result? The ED staff agreed to enter all results into the electronic medical record (thereby fulfilling the documentation of the first 3 items), and the fourth item would be documented in the handwritten ED medical record. The POC/ED testing program was implemented in early 2002. Audits of fingerstick glucose testing were performed in June 2002, September 2002, and December 2002. The documentation error rates ranged from 48%, 69%, and 44%. Multiple meetings, communications, and extensive retraining of the 111 ED staff/test personnel ensued. A subsequent audit in August 2003 revealed a 48% error rate in documentation. At this time, the clinical laboratory removed all POCT from the ED. After multiple meetings, the ED asked to have fingerstick glucose testing reinstated and agreed to improve their documentation. The clinical laboratory upgraded their glucose instruments to permit barcode identification of the test personnel (to eliminate a previously manual step), and reinstated POC fingerstick glucose testing in the ED in January 2004. An audit in February 2004 (1,057 tests) demonstrated a 91.5% compliance rate with the required documentation. An audit in March 2004 revealed a 99.5% compliance rate. Customer service, technical support, and regulatory confusion. Knowledgeable customer and expert technical support is essential. Such essential support is currently often not available, confusing, or contrary to regulation. Examples of this include: After CMS allowed the Equivalent QC (EQC) option, one manufacturer sent out valued customer notifications twice (06/09/04 and 06/30/04) stating that the new CLIA quality control rules (January 2003) affect those laboratories that hold a CLIA Certificate of Compliance and are inspected by CLIA surveyors. Laboratories that hold a Certificate of Accreditation, therefore inspected by an accreditation organization (AO) such as the CAP, the JCAHO, or COLA, are not subject to the CLIA changes but should check with their AO for any changes with the AO s inspection criteria. The person receiving this letter misinterpreted it and thought the manufacturer was broadly stating that any laboratory accredited by CAP or JCAHO was not subject to CLIA changes. These same 2 letters state that the manufacturer is committed to helping you implement a thorough Quality Systems Assessment (QSA)... A QSA maximizes your ability to detect system errors prior to reporting patient results. To meet CLIA regulatory requirements for reducing daily liquid external controls to once per month, an EQC Option 1 Implementation protocol can be obtained at the user website A subsequent 11/18/04 e-mail from the manufacturer s technical representative to a user states that the EQC Option 1 implementation protocol had not yet been prepared pending resolution of a disagreement with CMS s requirement that each and every analyzer be validated for EQC. The manufacturer is hopeful to have a subset approach accepted eg, a subset of one accepted since each analyzer that passes the Electronic Simulator test is equivalent to all others that have passed this test. This series of communications was quite confusing to the many laboratories receiving them as to whether the manufacturer had really developed a QSA program compliant with EQC. California Lab Field Services (LFS) surveys 5% of laboratories previously inspected by JCAHO, CAP, or COLA, and other laboratories as needed. Lab Field Services uses the State Operations Manual (SOM) to conduct their surveys. The LFS SOM is based on CLIA. One manufacturer s position on the need for California laboratories to meet EQC requirements is It is our understanding that a lab that is inspected by CAP, JCAHO, or COLA does not have to consider the EQC guidelines at this time. 10 This highlights a general confusion amongst clinical laboratorians in California facing a contradiction in which LFS requires compliance with CLIA requirements (including the EQC option, if applicable), yet a manufacturer was stating otherwise. Lack of recognition of user practices and effect on device 1 POCT instrument in widespread use in neonatal nurseries. Nurses typically obtain specimens using a 25 gauge needle, resulting in higher frequency of hemolyzed specimens. The manufacturer recommends use of 16-20 gauge needles only for specimen collection, but does not specifically state in the instructions that higher gauge needles should be avoided to minimize the risk of hemolysis. Nurses typically obtain 0.1 ml specimens in heparinized syringes for ionized calcium determinations. The relatively larger amount of heparin in these specimens results in binding of Ca ++ in the specimen and artificially lower ionized Ca ++ values. This information has not been widely distributed to users, and there is significant potential for therapies to be instituted or adjusted based on these incorrectly low ionized Ca ++ values. Reimbursement for POCT oversight what is that? This limited view into the POCT issues faced daily by a clinical laboratory director hopefully gives insight into the time and effort it takes to administer and appropriately oversee a POCT program. Testing aside, documentation of initial training, initial competency, and annual competency and management of the records alone for the 1,200+ test personnel is a never ending task. Fielding new requests for new POC tests not previously performed requires substantial technical and professional expertise. Of note, there is no separate professional reimbursement for this activity that easily consumes 10% to 15% of my daily effort. One benefit of a clinical laboratory oversight of POCT, however, is ready availability of expert technical help and advice and clinical consultation. There is also ready availability of definitive reference testing. These 2 factors result in optimum patient care. POLs. Outpatient clinics not on my campus are essentially POLs. While they have many of the same POCT issues as those overseen by my laboratory, there are additional issues related to Provider Performed Microscopic Procedures (PPMP). Most providers are not familiar with basic microscope maintenance, reagent quality control testing, or documenting storage temperatures (room versus refrigerator versus freezer) as part of an overall quality laboratory practice. When asked to participate in external image-based competency testing, many providers think that their professional expertise is being questioned and are personally offended. 624 LABMEDICINE Volume 36 Number 10 October 2005 labmedicine.com

During 1 survey of an outpatient clinic, 1 provider, when questioned about his incorrect response to a Gram stain image challenge, responded that he had been performing Gram stains for 30 years and knew what he was doing. When asked why he got the wrong answer, he refused to answer and left the room abruptly. At least 1 Web-based image competency testing program does not work on Mac computers (Apple), effectively decreasing compliance with annual competency testing for Mac users. In summary, the testing and quality issues surrounding POC and POLs are limitless. Recommendations Based on my experience, my recommendations for future POCT are: 1. For all unitized devices, include an internal control(s). 2. For all tests that require reading results in a specific time period, have the entire test area be completely unreadable or disintegrate after the specific time period has expired. 3. For all unitized devices include a simplified step by step test method on the cartridge itself. 4. For all instruments include lock out features for whenever test personnel identification is incorrect or patient information is not entered. 5. For all instruments include connectivity with medical information systems. 6. For any test device requiring specific storage conditions (eg, temperature, humidity) have the test area turn a different color or disintegrate if the recommended storage conditions have been exceeded. 7. For any instrument or test with a reportable range commercially available liquid QC must be available for the user to verify the reportable range. 8. Customer help lines should be knowledgeable and helpful with expert technical support readily available. 9. Electronic QC for instruments a nice touch to assure that the guts of the instrument are working as expected, but not sufficient by itself. 10. Require liquid QC to be performed as necessary to assure that the application of a substance of known concentration yields the expected result. Not a substitute for blood comparisons because the different matrix of liquid QCs. 11. Require at regular intervals that patient values obtained by POC be compared with those obtained by a reference method to detect any drift in POC testing not detected by electronic QC or liquid non-blood QC materials. A final word about EQC related to general Clinical Laboratory operations. A requirement of EQC is that once a test system has been validated and meets the requirements for EQC, then the frequency of QC can be dramatically decreased. The caveat is that if QC fails, any testing performed in the period between the last successful QC and the failed QC must be repeated. In my setting, a clinical laboratory serving an acute care hospital, this is not a reasonable option for at least 2 reasons. First, the average clinical laboratory does not have adequate storage space for all specimens tested in the maximal interval allowed by EQC between QC runs that may require retesting if the next scheduled QC run fails. This is not an insurmountable problem, in that specimen storage could be minimized to coincide with the current storage capacity in the Clinical Laboratory. As an example, my clinical laboratory holds EDTA-anticoagulated blood sent for complete blood count (CBC) analysis for 3 calendar days. I could review the QC on my automated hematology analyzers, validate them for EQC, and if validated, reduce the frequency of QC to once every 3 calendar days. The second and by far the most important reason why EQC is not applicable to a clinical laboratory in an acute care setting, however, is that laboratory results are typically used within hours (if not minutes) of their release to guide patient therapy. The rapidity with which these results are used does not allow any test system to be out of control for any period of time. LM 1. Goldschmidt HMJ, Lent RS. Gross errors and work flow analysis in the clinical laboratory. Klin Biochem Metab. 1995;3:131-140. 2. Astion ML, Shojania KG, Hamill TR, et al. Classifying laboratory incident reports to identify problems that jeopardize safety. Am J Clin Pathol. 2003;120:18-26. 3. Steindel SJ, Granade S, Lee J, et al. Practice patterns of testing waived under the Clinical Laboratory Improvement Amendments. Arch Pathol Lab Med. 2002;126:1471-1479. 4. LaBeau KM, Simon M, Steindel SJ. Quality control of test systems waived by the Clinical Laboratory Improvement Amendments of 1988. Arch Pathol Lab Med. 2000;124:1122-1127. 5. Goldschmidt HMJ, Lent RS. From data to information: How to define the context? Chemometrics and Intell Lab Sys. 1995;28:181-192. 6. Foucar K. University of New Mexico; personal communication, 2003. 7. Woo S. Point of care coordinator, Kaiser Permanente Medical Center, Walnut Creek CA; personal communication March 2005. 8. Charache P. Johns Hopkins University, Baltimore MD, personal communication, 2001. 9. Ng VL, Kraemer R, Hogan C, et al. The rise and fall of i-stat point-of-care blood gas testing in an acute care Am J Clin Pathol. 2000;114:128-138. 10. Chapman G. Abbott Point of Care; personal communication, 2004. labmedicine.com October 2005 Volume 36 Number 10 LABMEDICINE 625