Q u a l i t y a s s u r a n c e i n d e n t i s t r y : e x e c u t i v e s u m m a r y, p a r t 3

Transcription

1 Q u a l i t y a s s u r a n c e i n d e n t i s t r y : e x e c u t i v e s u m m a r y, p a r t 3 A D A Quality Assurance Project I n publishing the final segment of the Executive Summary of Quality Assurance in Dentistry, the direction for future quality assurance activities in dentistry is presented. In Parts 1 and 2, descriptions of the basis for and methodology of this study were given, along with the data collected. Part 3 focuses on the practical considerations by taking the information assembled and putting it to use in dental practice. This final part identifies the recommendations for future research and the policy considerations that evolved from the study, using all the relevant, available objective material and subjective comments from the many consultants to the project. The recommendations are presented, using the major topics from the Quality Assurance Model for Dentistry, which lists 97 characteristics in 11 subject areas common to all quality review methods. By looking at each of these topics, it was possible to identify the gaps in information and consequently to determine where further study would be beneficial. The final task of the project was the development of the three field test models. These are templates for research studies; three different approaches to quality review are used. Because each system had been used in a particular setting, each had to be modified to fit the special needs of a variety of practice settings, in particular, the solo private practice. Although the Executive Summary presents the key issues and findings of the study, if a more in-depth examination of the subject is desired, the two-volume final report, Quality Assurance in Dentistry, is available through the Council on Dental Care Programs of the American Dental Association. R e c o m m e n d a tio n s for p olicy co n sid era tio n and further research n eed s Analysis of 11 quality review systems and discussions with quality assurance experts and dental practitioners have raised many questions about the future of quality assurance in dentistry. The project staff has prepared a list of recommendations for policy considerations and further research needs. The recommendations are presented according to the components of the Quality Assurance Model for Dentistry. A fuller understanding of the basis of these recommendations is found in Volume I of the final report. P la n n in g Policy considerations. Appropriate incentives to encourage dentist participation in quality assurance activities should be identified. Providers needs and perceptions relative to these activities require assessment. A budget is an important characteristic of a quality assurance system. The exclusion of a budget precludes another important item, determining the system s costeffectiveness. A planned and monitored budget should be included so that costs can be weighed against benefits accrued. Then it will be possible to identify the best results obtained for the amount of money expended. Further research needs. Quality assurance activities in solo practices have been limited, and the applicability of existing systems to this setting should be explored. Replication is needed in a variety of solo practices. Relationships between quality assurance and continuing education programs should be explored. Research by Weinstein and others1 indicated a significant relationship between quality of care and perceived usefulness of university courses and study clubs. Further, participation in quality assurance activities should be considered a form of continuing education, and provisions for credit should be made. A d m in istra tio n o f the system Policy considerations. Examination JADA, Vol. 98, M arch

2 and discussion are needed to determine what types of information should be documented and who should be required to see review reports in a quality assurance program. Then specific guidelines to assure confidentiality of reports and the completeness of the review cycle must be developed before review and should be made known to all who are being reviewed. At the time of implementation, a review system should have specified frequencies of occurrence, and the integration of the system in the practice structure should be defined clearly. Regularly scheduled review that is well integrated into office activities enhances the review s importance, whereas an unstructured, haphazard approach may not. T o p ic a n d p r o v id e r se le ctio n Policy considerations. Review systems should be able to accommodate assessment of all types of dental care practitioners. Further research needs. Separate studies by Brook and others2 in medicine and Bailit and others3 in dentistry show that there may be little variability in the quality of health care services within a practice. Identification of the quality of the most frequently provided services may be indicative of the quality of all services. Studies are needed to determine the variability within a dental practice among treatments rendered. In this way, a sample of cases for which sufficient data can be gathered to yield meaningful results will be identified. C a s e se le ctio n Further research needs. Although it may not be practical to select a genuine random sample for each audit, the sample should be representative of the quality of care rendered in the practice. An expedient, valid selection process should be developed to provide the most meaningful results. The variability within a practice in performance of a single treatment or in treatment planning needs to be identified so that an appropriate minimum sample size can be determined regardless of the approach to review. C r it e r ia a n d sta n d a rd s Policy considerations. Until a national system is identified, thoroughly field tested, and found to be an effective and acceptable mechanism for review of the quality of dental care, criteria should be developed locally for each system. These will serve as sample criteria for the adequacy of the system in achieving its goals. However, criteria need to be established for all three principal approaches to review, particularly those which can be applied to profiles or aggregate data and those for record review. Valid clinical criteria already exist in several programs. An important issue related to the quality of dental care is underutilization: Although many of the systems reviewed did look at this aspect of quality, only those that operate in highly defined populations could adequately deal with the problem. Most systems concentrate on the users of services and do not address those who are not. Efforts should be made to obtain utilization data whenever possible through dental prepayment administration, Medicaid programs, and others. The total eligible population in an area can be identified, and those who obtain services can be compared with those who do not. Provisions should be made in a review system for testing the reliability and validity of criteria and standards. Further research needs. A greater effort should be made to examine criteria development based on a topic-specific approach, particularly for record reviews. Increased utilization of nondentists will be necessary to conduct review to the extent possible in all areas where clinical judgments are not required. The decisions relative to selecting a process of care require professional judgment. Efforts should be directed to developing and testing branched criteria for dentistry that reflect decision-making. Branched criteria are structured to reflect the decision-making process in developing and performing a treatment plan. Depending on circumstances in each case, decisions are deemed acceptable or unacceptable. Outcome review has several drawbacks, including the influence of outside factors such as patient compliance, systemic disease complications, and the fact that real outcomes may not be realized for many years. Emphasis should be given to the assessment of process and shortterm outcome and to the examination of the capability of these approaches to truly reflect the quality of dental care and to identify basic problems in care. PSRO law calls for the review of the necessity and appropriateness of care. Most dental quality assurance programs were not developed to comply with PSRO regulations. As a result, these two areas have not been addressed explicitly in current programs. Necessity and appropriateness of care should be priority items of criteria development, and the topic-specific approach should be considered a vehicle to carry this out. The development of national sample criteria should be completed after aspects of care related to necessity, appropriateness, and quality have been identified and systems and data sources to provide the desired information have been proved adequate. D a ta so u rce s a n d data co lle ctio n Policy considerations. Several disadvantages of clinical evaluation as a primary method of quality assurance have been documented. The cost of implementing a clinical evaluation program can be prohibitive because examinations must be performed by dentists. It also is difficult to use a random sample of cases as cooperation of patients cannot be guaranteed. Clinical evaluations are also most likely to disturb existing patient-dentist relationships than are other methods that do 4 30 JADA, V ol. 98, M arch 1979

3 not directly involve the patient. Because of these disadvantages and because of the extensive effort already expended in this area, the clinical evaluation approach to a quality assurance system should not be a priority. However, clinical examinations are valuable in determining the technical quality of care, and they should be reserved for those cases in which other methods of review cannot judge quality effectively. The state of dental records needs examination, and the essential quality assessment components of the record need to be identified. If record audit will require changing the record-keeping practices of the profession, consideration should be given, at least initially, to another existing data source, namely the claim form. The costs associated with the profile type of review may be the most practical for first-level screening to determine problems in ambulatory dental settings. The profile review does not require travel by dentists; data are collected and transmitted to a central point for processing. Profiles should not be considered the sole method of review but should serve to identify treatments, conditions, or providers that require in-depth review. Record review, pending identification of auditable records, should be used as the next level of review and should use explicit criteria to make review objective. If the question of quality is still not fully resolved, clinical examination may be necessary. If several approaches to review are considered, integration of the information is required. Each approach may serve as a problem identifier for the others. Targeting review to identified problems provides an efficient way to determine the real cases of inappropriate care. Review can be targeted or focused on the identified problem areas, and other areas of dental care can be reviewed less frequently. Further research needs. Data utilization or profile analysis based on encounter or claim forms should be one area of priority in dental quality assurance research. The following factors should be considered: Correlation of quantity information and the assessment of the process and outcome of care through other forms of review. Identification by peers and PSRO of the most meaningful types of profile reports for assessing the quality of care. Formulation of a minimum data set that can be gathered uniformly in all types of settings for the purposes of quality assurance. Data collected on the uniform claim form, approved by the American Dental Association, should be a starting point for such a data set. Other data that can provide assessments of outcome should particularly be stressed. Any field test should build in comparable studies that use different approaches to the same cases so that the various methods of review will be validated. A n a ly s is a n d p e e r re v ie w Policy considerations. Documentation of decisions relative to profile analysis should be required. The report can serve as a feedback mechanism to enhance the understanding of review decisions. Because of the desirability of review by peers, review committees should be established in those settings where peer review is not possible. In solo practices, review committees can be established through a consortium of solo practitioners. Systems that are selected for implementation must provide methods to control for reviewer bias. F e e d b a c k, a ctio n, a n d a p p e a ls Policy considerations. Provider acceptance of audit activities is a desirable and necessary goal. One means of gaining acceptance is to base the review system on provider input and to encourage provider participation in review activities. A provider is less likely to fear and oppose a system in which he is involved. The current attitude of anger at criticism will be improved if it is possible for the provider to improve his care as a result of review. If educational programs are incorporated into the review system, review can become a learning experience for the provider rather than being simply a punitive monitoring technique. Many of the providers in the systems reviewed thought that ingrained review activities would eliminate many of the malpractice actions that health providers experience today. This could be another motivating factor for review participation. Further research needs. The way in which the results of review are presented to the providers being reviewed should be examined further. It should be determined if it is more effective to demonstrate one dentist s practice patterns across many conditions or to compare his practice patterns with those of similar practitioners. The goal is to change aberrant behavior, and the way in which findings are presented may have a significant effect on obtaining desired behavior changes. F o llo w -u p Further research needs. Monitoring provider behavior change is the least-developed facet of the followup methods reviewed. Research is needed to develop methods of gathering the data to check changes in provider behavior over time. Methods for mini-audits in noninstitutional settings also should be developed. Initiation of valid procedures for follow-up audit of a small number of cases helps to eliminate subjective, one-on-one verification of behavior change. E v a lu a tio n o f th e r e v ie w system Policy considerations. Although most of the programs regularly reviewed system components, in many cases the review process was neither structured nor documented. Efforts should be directed toward developing an ongoing system to review the components of the quality assurance program itself to ensure that the methods and procedures QUALITY ASSURANCE 431

4 used are effective in achieving the program s goals. Im p a c t o f re v ie w Policy considerations. Because an extended time frame may be needed to identify changes in provider behavior patterns or in oral health status, long-range studies to evaluate the impact of a quality assurance program on provider behavior and oral health status should be planned at the outset of the program. The complexities in examining the cost-benefit relationship make it necessary to look at review results over time, aggregate the findings, and place some value on them. This value can then be related to costs of conducting review. Benefits of review in terms of improved efficiency in the use of dental resources and manpower should be weighed against the cost of review. Further research needs. Development of appropriate measures of patient satisfaction is needed. The problem of biased respondents must be addressed to more fully assess the impact of quality assurance activities on patient attitudes. There are standardized measures to review oral health status, for example, decayed, missing, filled teeth ratio, Oral Hygiene Index, and Plaque Index. The feasibility of using these measures to evaluate the impact of quality assurance activities should be explored. Indirect outcome measures should be developed, such as the Index of Need or stage of care for assessing oral health status. Field test protocols Selection of models of quality assurance for field testing was accomplished through reviewing scores received using the Quality Assurance Model for Dentistry, comments on the systems from project consultants, and the systems ability to address the research priorities identified in examining the state of the art. The field tests are intended to test not only the practicality of implementing a system of quality assurance, but also to test and examine some of the research recommendations identified in the study. The objective of the current research is to identify models of quality assurance that are consistent with the PSRO format; thus, the proposed protocols focus primarily on appropriateness and necessity of dental treatment. Two of the systems examined in the study were selected for field testing. They were developed by the University of Pennsylvania Department of Dental Care Systems, School of Dental Medicine, Leonard Davis Institute of Health Economics and the National Health Care Management Center, Philadelphia, and Drs. Neal Demby and Murray Rosenthal for the Sunset Park Family Health Center, Brooklyn, NY. A third program of quality assurance proposed for field testing is based on a model currently used in medicine by the Utah Professional Review Organization and is known as the Physician s Ambulatory Care Evaluation (PACE) program. The program is modified for dentistry. The three projects would be monitored by a national advisory panel composed of practicing dentists, consumers, and experts in quality assurance. It is hoped that through consistent evaluation of three field tests of dental quality assurance, the national advisory panel would be able to identify the strengths and weaknesses of each and to provide answers to many of the questions facing quality assurance in dentistry. Field test 1 The University of Pennsylvania staff has developed an overall resource management and quality assurance methodology that can be used in a variety of settings. The computer technology needed to implement the system has been created, as has a data set specifically designed for ambulatory dentistry. A modifica-^ tion of this system of dental assessment serves as one model that will be tested as a viable approach to review. O b je c tiv e s o f th e p ro je c t The objectives are: To continue the development of a practical and effective method of quality assessment and quality assurance. The project has been designed to test and modify as necessary the system of quality review developed at the University of Pennsylvania by the Department of Dental Care Systems. To further develop profile or pattern analysis as a viable, costeffective method of review. To evaluate the proposed quality assurance system in terms of effectiveness, impact of practitioners, and practice structures and cost of operations. To propose more widespread applications of the system to various practice, quality assurance system, and payment mechanism structures. Included in these recommendations would be the adaptations or modifications necessary to make the system applicable in these various settings. P ro je c t d e scrip tio n This field test proposes that a threetiered system of quality review based on an adaptation of the University of Pennsylvania system be instituted in Worcester, Mass, under the administration of the Central Massachusetts Health Care Foundation (CMHCF) as a potential prototype for future large-scale review systems. First level. Profile or pattern analysis. Encounter data from dental offices in the Worcester, Mass, area will be analyzed by project computer staff through comparison of practice profiles. Approximately 40 practitioners will participate through submission of encounter data on the care of approximately 300 patients each. Profiles, together with written commentary on their possible or probable significance, will be forwarded to a Dental Quality of Care Committee (composed of dentists in the Worcester area) for review and further action. Second level. Focused nondentist 432 JADA, Vol. 98, M arch 1979

5 review through data abstracted from selected dental charts. Based on the most plausible interpretation of profiles, the project computer staff will generate reports that will form the basis of a targeted or focused second level of review. These reports will identify specific practices, practitioners, and patient charts, and will explicitly specify what types of information a nondentist reviewer will abstract from the records. The Dental Quality of Care Committee will help define the types of information that the auditor will abstract. In these reports, records will be selected in response to specific diagnoses, treatments, outcomes or patterns of care that are suspected to be problem areas. The auditor will thus have specified charts to audit for patients who have the same set of conditions in reference practices and for practices in which problem areas are suspected. The reference practices would be the ones that profile analysis suggests are delivering care at an above average level of quality. The results of the audit data from the nondentist assessor will be forwarded to the Dental Quality of Care Committee for review and further action. Third level. Peer review by chart audit or direct clinical examination. or both. When the Dental Quality of Care Committee is unable to make judgments about relative quality, or has insufficient information upon which to formulate a quality judgment, members of the committee will act as assessors to audit dental charts and perform direct clinical examinations. As in the second level review, the assessors will compare reference practices and practitioners with those that are suspected as being aberrant. Thus, in none of the three levels of review is the assessment general or diffuse. The profile is used to identify a specific suspected problem area, and the second and third levels gather information to permit the Dental Quality of Care Committee to determine if, in fact, a problem of poor quality exists. The technique of comparison of practitioners at different points in a range of performance is used. Field test 2 A protocol has been developed to test a second quality assurance system. The system, which has been operating for over five years at the Sunset Park Family Health Center in Brooklyn, NY, was developed jointly there by Drs. Neal Demby and Murray Rosenthal. The proposed field test involves implementation of the management information system (MIS) and clinical examination portions of the Sunset Park system in the private practice solo and group offices of 30 general dentists in a rural area and 30 in an urban area of the same state. O b je c tiv e s o f the p ro je c t The objectives are: Test the effectiveness of the Sunset Park system in effecting change in practitioner behavior and improving both the quality of dental care provided and the oral health status of patients. Test the feasibility of implementing this or a similar system in the private sector. P r o je c t d e scrip tio n In the initial phase of data collection, a data abstractor will abstract encounter information for approximately 30 patients of each participating dentist. The patients will be selected from those who have received continuous care from the dentist for the previous five-year period. Clinical examinations of the same patients will be conducted at the participants offices by two dentists, one from the local dental society s Peer Review Committee. The examination will also include a review of the record, radiographs, and study models to determine the necessity of care and the appropriateness of the treatment plan. Outcome of care as determined by the clinical evaluation will be compared with the process of care as determined by the patient and provider profiles developed from the encounter data that demonstrate service rates and treatment patterns. Through this comparison of cases, a correlation of process of care and outcome of care can be considered. Further, the validity of profiles as a method of assessing quality of care can be examined by correlating the results of review using indirect measures and the results of review using direct examination of the patient. Each participating dentist will receive immediate feedback on this initial test and will have an opportunity to relate the process of care he has provided for five years to the current oral health status of the patients. In order to examine the impact of the system on dentist behavior and quality of care, encounter data will be collected for an additional 18 months on the initial sample of patients plus a sample of new patients. The participating dentist will receive monthly computer-generated reports on the care they have Tendered. Group statistics will also be sent to the peer review committees. This committee is then responsible for analysis, action, and follow-up. At the end of the 18-month data collection period, a second series of clinical examinations will be performed. Analysis of the results of the field test will focus on the impact of the system on the quality of care and on the cost of implementing the system. Other important aspects of the field test will be the comparison of the results of measuring the process of care with those of measuring the outcome of care, testing the validity of using profile data as a source of quality information, and testing the feasibility of expanding the role of component and constituent dental society peer review committees to include ongoing quality assurance activities. Field test 3 The claim form is a principal data source for third party review and has proved to be an adequate data source QUALITY ASSURANCE 433

6 for screening purposes in medical care evaluation. For several years, the Physician Ambulatory Care Evaluation (PACE) program has used claims data and an advanced automated system for utilizatioivand quality review purposes. Because of its success in the medical sphere, the PACE system has been selected for exploration as a model for ambulatory dental care evaluation. The PACE system is of particular interest because it is consistent with the intent of PSRO, the suggestions of project consultants who are in private practice, and the recommendations of this study. Most existing dental claims processing review systems focus primarily on utilization, fraud and abuse detection, and benefit coverage. The PACE methodology, however, applies a more sophisticated computerized system of review that uses criteria for care developed by expert medical specialists to evaluate appropriateness of treatment provided as an aspect of quality. A physician s care that repeatedly varies from these peer expectations is examined more closely. An essential feature of the PACE system is that it controls for claims form limitation as a data source by focusing on patterns of care rather than individual exceptions. Screening criteria are designed to accept a wide range of clinical practice and circumstances and to consider many variables, including procedures, types of patients, provider attributes, and treatment time. The PACE system not only considers these variables as they appear on an individual claim form, but it also utilizes the memory function of the computer to look backward and flag forward a predetermined length of time to evaluate an entire sequence of treatment. PACE criteria may be classified into five types: Required service or therapy. Contraindicated service. Prerequisite history needed to justify the service or therapy. Utilization limit for service or therapy. Untoward symptoms during therapy. A d a p ta tio n o f the P A C E system to d e n tistry Because automated claims-based review of quality using the PACE logic is a new concept in dentistry, the model will be developed and implemented in two phases. The first phase will address the issue of whether functional screening criteria can be developed to evaluate the quality and appropriateness of dental care using claims data. If functional criteria can be identified and the criteria prove sensitive to claims data, the second phase of the project will be to integrate the model into one or more existing claims-processing systems and test the cost benefit of various methods of selecting claims for posttreatment screening. The purpose of the testing will be to identify criteria and a sampling methodology that focuses on problem cases, the product of which will be a more efficient, objective device for selecting cases for in-depth analysis. P h a s e I Goals and objectives. The overall goal of phase I is to develop a theoretical model for quality review using dental claims data. This larger goal may be broken into seven objectives. Objective 1. Develop screening criteria for automated quality evaluation of dental care using the PACE logic. Objective 2. Identify appropriate data sources. Objective 3. Program criteria into a software package and test them. Objective 4. Review test results to determine if PACE logic is functional for dental quality review, and make indicated criteria revisions. Objective 5. For each criterion, determine what, if any, additional information is needed to make treatment recommendations, make hypothetical treatment recommendations, and compare the results with actual recommendations for care. Objective 6. Develop additional guidelines for use by nondentists in manually selecting exceptions and gathering information for peer review. Objective 7. Make recommendations for testing the model in phase II. P ro je c t d e scrip tio n In establishing a set of criteria for use in automated quality review, a panel of practitioners will be asked to keep two considerations in mind. Does the information provided by the criterion clearly indicate appropriate practice? If the information is not a clear indicator, what additional information is needed? Actual claims data will then be screened against the criteria, and all exceptions to the criteria will be examined. In determining if a criterion is a sensitive indicator, a committee will consider the volume of exceptions for each criterion, the significance of the identified problem on oral health, how well the screening process results reflect the quality of treatment, the contribution of the system to existing review mechanisms, and whether aberrant behavior identified in the system could be altered. The system will be considered applicable if the data generated would be used to alter aberrant behavior. Another major objective will be to determine if there are any patterns of exceptions so that different methods of sampling claims for retrospective automated review can be identified which will efficiently focus on the problems (by cost, by services, by provider). Also, decisions made via computerized screening review will be compared with actual decisions made via routine pretreatment review to identify areas that may be more efficiently examined through the PACE type of review. 434 JADA, Vol. 98, M arch 1979

7 P h a s e I I Phase II will be used: To identify the most cost-efficient application of the model to pretreatment review. To identify one or more sampling methodologies for selecting claims for retrospective review. To determine the impact of the system on practitioner behavior. These will be accomplished by running the project on a much larger data base on an ongoing basis. Exceptions will be reviewed by a peer committee; education and actions for problems identified will be implemented. At the same time, comparisons of the volume of real problems identified and costs of identifying problems will be compared among the various sampling methods tested. Finally, the three field tests proposed use profile analysis as a principal mechanism for review. The profile-based method as an approach to review was chosen to be field tested for two reasons: its anticipated cost-effectiveness and the availability of a uniform data source. Because dental researchers have found that patient records provide little more information than claim forms, testing a detailed record examination system did not seem advisable at this time. However, the project staff and Core Consultant Committee did feel that record review, particularly as a second level review, is important and needs exploration. The most comprehensive approach developed to date is that proposed by Dr. Seymour Roistacher, who identified a topic-specific approach to record audit using peer-developed, explicit measures based on the medical care evaluation (MCE) study model used in the hospital setting, and further examination of this approach in ambulatory settings is recommended. Dr. Roistacher suggests these areas to study: Workup: Was the workup appropriate? History: What constitutes the necessary information that should be documented in the patient s history before treatment? Examination: What should be noticed in the examination for the patient s condition? Laboratory: What laboratory test findings are needed before treatment? Elements essential to management: What items of management are essential for treatment of the condition? Complication: What complications might occur frequently and, if found, should be reviewed? Condition at discharge: What is the optimal condition at discharge? (This may include instructions to patients.) All three field tests will be fully developed, and funding will be sought to perform them. This report was prepared by Susan Kahn Stem, Suzanne C. Morrissey, and Jennifer Mauldin. 1. Weinstein, P., and others. Quality and perceived usefulness and utilization of continuing dental education. J Am Coll Dent 44(4) , Brook, R.H.; Williams, K.N.; and Avery, A.D. Quality assurance today and tomorrow: forecast for the future. Ann Intern Med 85(6) , Bailit, H.L., and others. Quality of dental care: development o f standards. JADA 89(4) , QUALITY ASSURANCE 435