The Netherlands. Tulips. Cows

Transcription

1 Guidelines in context Implementing guidelines and the role of clinical audit Prof. Richard Grol Center for Quality of Care Research (WOK) Netherlands

2 The Netherlands Tulips Cows

3 Best approach to improving performance: a tale of two worlds World of evaluation control and incentives Externally driven QI Accountability Transparancy Reliability Top-down External incentives Management focus Etc World of EBM, guidelines and education Internally driven QI Trust in professional Self-regulation Validity Bottom-up Intrinsic motivation Professional focus etc

4 Putting guidelines in a wider context Clinical guideline development has no value in itself, the only questions that matter are: -how to effectively improve the care for our patients? -(to what extent) can clinical guidelines contribute to that?

5 The success of guideline development EBM, research syntheses and guideline development major success in health care Worldwide collaboration (Cochrane, GIN, etc) Guideline development programs in most developed countries, many use AGREE New interesting scientific movement related searching, grading, summarizing evidence and developing clinical guidelines New world of clinicians, epidemiologists, guideline experts, own meetings, procedures, vested interests and organizations

6 Assumption: all these guideline development activities will ultimately lead to better patient care! Is this assumption justified? Maybe yes, but:

7 Possible problems with guidelines Guideline development is also expensive Substantial bias in systematic reviews and clinical guidelines: final guideline recommendations are (partly or largely) reflection of national or local culture or personal views of developers Evaluations of guidelines: often do not meet quality criteria (AGREE) Even well developed, evidence based recommendations are often not used in clinical practice (30-40%) Most efforts to implement guidelines only moderately successful in improving patiënt care and outcomes

8 Treatment of heart failure (front page National Post, Canada 14 Sept 2005) Canadian study (JAMA) among more than 1400 heart failure patients showed that: Low risk patients (14% chance of dying within year) were 61% more likely prescribed ACE inhibitors and 80% more likely to be put on beat-blockers than high risk patients (45-50% chance of dying within year)

9 A major underestimated problem in health care: knowledge about optimal care is not applied Given that so much (knowledge on optimal patiënt care) is lost at the real world level, as has been demonstrated time and time again for simple research outcomes, I believe that our approach to patiënt oriented research should be reconsidered Lefant, New Engl J Med 2003

10 Being the devils advocate Critical policy makers, applying same criteria as used in reviewing scientific evidence for patiënt treatment to procedures and outcomes of guideline development, may ask questions: Why spend so much time and money on something that is so time consuming and expensive and nevertheless still so biased, badly used and ineffective in improving patiënt outcomes? Can I get better value for my money? Do we need more external control, transparancy and incentives to improve care?

11 Comprehensive handbook on implementation of guidelines and improvements in health care (Elsevier 2005)

12 What is required for effective implementation of evidence in patiënt care? Specific (evidence based) goals for patient care, in attractive, understandable and accessible format Valid indicators to measure quality: rigorous development based on evidence and testing in practice Reliable and acceptable instruments and procedures to collect data on practice performance Effective, accessible, acceptable systems for feedback and reporting to providers, authorities,and public (Cost-)effective quality improvement programmes adapted to target group, support to target group Continuous monitoring of progress

13 The innovation: how to develop an attractive message? Gladwell: how to assure that messages stick in memory of potential users and stand out in overload of information? Message on evidence or guideline should create sense of urgency: not only content, but also format/ presentation are very important Structuring, packaging recommendations into practical, personal advice: evidence summaries, local protocols and care pathways, clinical indicators, decision aids, patient information leaflets Budget for dissemination needed to develop such messages and packages

14 Assessment of quality of antibiotic prescribing for pneumonia end COPD in hospitals (Schouten et al. WOK 2006) Indicators derived from (inter)national evidence based clinical guidelines, consensus by rigorous Rand Delphi procedure, empirical test on 19 indicators and 899 patients in 8 hospitals: For 3 indicators no suitable data in medical record It was possible to collect reliable data for other indicators; two independent assessors: kappa >.60 No room for improvement: one indicator had a score > 90% Study of impact of age, sex, severity of disease on indicators: case-mix corrections needed for two indicators 15 indicators in final set

15 Quality of antibiotic prescribing for patient with CAP in hospitals ( Schouten et al WOK JAC 2005) Assessment of patient records in 8 hospitals (evidence levels): - Timely start with AB therapy (B) 68% - Prescribing AB on indication (B) 45% - Timely switch IV to oral medication (B) 81% - Timely switch from broad to - small spectrum (C) 80% - Two sets of blood samples for culture (B) 57%

16 Intervention to improve AB use Complex intervention linked to analysis of barriers (qualitative interviews with clinicians): Structures: local coordinator and improvement team Education: educational sessions with opinion leaders, local consensus Feedback: individual feedback and peer comparison, discussion of feedback in team Reminder/decision support Process analysis and redesign for obtaining cultures

17 Effects of intervention Cluster randomized trial in 8 hospitals with baseline and follow-up measurement; difference in change between I and C : Timely initiation +24% AB use on indication + 2% Switch from IV to oral - 8% From broad to small spectrum -12% Two sets of blood samples +4%

18 Improving AB use in hospitals: observations of a change agent Physicians, pharmacists and microbiologists were well informed about the project and said that it was important to improve routines. But they often did not meet the arrangements made and wanted to keep working as they did, particularly the physicians Educational meetings were sometimes seen as breaks, physicians in some hospitals were only motivated in case of direct profit. Hierarchy was not made explicit and not everyone was taken seriously: the specialist was at the top, quality staff and head nurses at the bottom. Most QI-work was delegated to nurses

19 Lessons learnt about changing clinical practice good change proposal, with an attractive message is the start valid data on performance crucial to define concrete targets for change and provide feedback to providers and others mostly well planned change interventions needed, tailored to data on performance, specific problems found and specific features of target group and setting sometimes simple actions, but mostly actions at different levels, aimed at professionals, teams, patients, organizations actions partly educational, partly coercion and control organizational and structural measures, resources, support, training mostly before influencing individual decision making most of time receptive environment needed: positive attitude to and a culture of change in professionals and teams

20 Measurement of quality: confusing concepts and aims Different concepts: indicators, criteria, standards, accreditation, certification, assessment, clinical audit, pay-for-performance, physician profiling, public reports,etc. Different aims: feedback and improvement of practice, transparency, control, more choice for patients, licensing, financial incentives or sanctions, etc.

21 Definition of indicator A measurable element of practice performance for which there is evidence or consensus that is can be used to assess quality and hence change in the quality of care provided (Lawrence 1997, Campbell 2002) An indicator does not make direct or absolute statements about quality but functions as a signal that something may be wrong and that more detailled analyses are needed

22 Indicators Should be valid and reliable: measure what they should measure in a consistent, reproducable way They can be used to distinguish between hospitals, practices, providers that deliver better and worse quality They are sensitive to change: can measure improvement It is possible to derive reliable data for those indicators in a feasible way They are understandable for those who use the results They are acceptable for those who are measured Conclusion: rigorous procedure for development and testing needed!! Most indicators used now have not been rigorously developed and tested in practice!!

23 AIM OF QUALITY MEASUREMENT: WATCH DOG OR LAP DOG?

24 Different parties have different aims in measuring quality Aimed at internal quality improvement: evaluating if improvement is possible and needed, and initiating and assessing change, by professionals or health care institutes (self-regulation) Aimed at external control: accountability and transparancy to society (performance indicators) -to patients: to provide information to compare care providers and improve choice by public reports -to inspectorate: guarantee of safe care that meets minimum requirements of quality -to payers, insurers: for negotiating contracts based on price-quality arrangements -to authorities: guarantee that health care is good

25 Trust or control? Quality improvement and evaluation of care as a professional responsibility: emphasis on autonomy, intrinsic motivation of professionals to develop their own standards and guidelines and to improve care Versus Quality improvement and evaluation as a societal responsibility: emphasis on accountability, transparancy, external standard setting, evaluation and control, public reporting, financial incentives

26 Development in thinking about quality improvement In the 80s and 90s mostly self-regulation by professionals or institutions (education, clinical guidelines, peer review, clinical audit, etc) Next more emphasis on systems : improvement of organization of care, of care processes (managers in lead, TQM, accreditation programs, etc) Now: emphasis on top-down programs of external evaluation, transparancy, financial incentives for quality, initiated by authorities, purchasers and consumer organizations

27 National weekly: the best hospitals and specialists in 2005

28 Evaluation, accreditation and certification Accreditation is mandatory requirement for hospitals in Scotland (CHAI), France (ANAES) and Italy Obligation to introduce quality management programs and participate in external quality assurance activities (Germany, Switzerland) Obligatory performance measurement, indicators and public reporting in hospitals in UK, Netherlands, Denmark, Sweden, and other countries Performance indicators and public reporting in Medicare, Medicaid and many health plans in USA Pay for performance models in USA (over 160 experiments running), UK and (experiments in) some other countries

29 Pay for performance: GP-contract in UK Family doctors have new contract per 1 April 2004: about 20-25% of income related to quality of care, that is 1050 credit points linked to set of indicators (clinical performance, patiënt experiences, practice management) For an average practice (3 doctors,5500 patients) max pond as a quality bonus per doctor Evaluations of impact: increase in ICT and EPR, more tasks to nurse, more disease management of chronic patients About 91% of maximum score achieved; 83% of patients received recommended care; system much more expensive than expected Exclusion of patients from measurement possible, some strategic behaviour found

30 Trust or control: the debate Debate in scientific journals: No system of external measurement and auditing will be able to substitute for the relations of trust and professionalism which can also promote quality (O Neill 2003) This is the fundamental paradox that we shall have to embrace: that the trust between ourselves and our patients on which we depend can only be protected by strict enforcement of tough and demanding standards (Jewell 2005)

31 tendency towards more control leads to a virus of datafetisjim, a diarrhea of assessment mechanisms professional autonomy and freedom is pure ideological kitsch, explicit criteria for appropriate performance are indispensible control can prevent disasters but does not create value a culture in which trust and freedom of decisions are central stimulate motivation and productivity human beings are inclined to do good, however they sometimes fail to do so it is normal now that everybody also scientists, teachers and ministers- account for their performance

32 (Dis)advantages of external evaluation and control Pro Society has a right to receive correct information on performance Trust can only be maintained when care providers are open about their work Without strict demands of transparancy certain care providers are resistant to change Systems that are based on just education and intrinsic motivation have insufficient effect Contra Difficult to develop valid indicators for performance May lead to strategic behaviour, manipulation of data Expensive system needed to control data,more bureaucracy Effects of external evaluation and performance indicators unclear Potential side effects of external evaluation: loss of motivation in professionals, loss of confidence of public

33 How to bridge gap between worlds of trust and control?

34 Accreditation system for primary health care in the Netherlands: process Three-yearly (voluntary) full assessment Feedback by visitor: plans for improvement Trained assessors check working on improvement plans and progress made Accreditation based on full cycle of assessment and improvement

35 Accreditation system for primary health care in the Netherlands: topics Clinical performance (clinical indicators derived from national evidence based guidelines) Prevention (idem) Practice organization and management (consensus derived indicators, validated instrument) Patients experiences (validated instruments)

36 Methods Panels representing physicians, insurance companies, and patients determined a set of 81indicators (medical performance, practice organization and management, patients experience) Methodology used: -use of existing indicators and instruments -modified Delphi procedure with questionnaires and panel meetings -test in practice -patiënt feedback

37 Clinical indicator development in primary care (Braspenning, Grol et al, WOK 2004) all 70 national (evidence based) clinical guidelines available panel of 8 GPs: appraisal of key recommendations on relevance for health of patients and for costs results: 139 indicators from 60 guidelines data collection in 101 practices (almost 200 GPs) with patients results: valid and reliable data on about 60 indicators

38 National audit on clinical performance in primary care (Braspenning, Grol et al, WOK, 2004) Audit in 100 practices (National Network Primary Care), using 57 rigorously developed clinical indicators derived from 60 national evidence based guidelines for primary care: following guidelines total 74% prescribing decisions 62% referral 86% diagnostic testing 71% preventive care 78%

39 Flu vaccinations in primary care in the Netherlands in 2001 (WOK data National Information Net GPs) Percentage vaccinated: all high risk patients 76% > 65 years 81% diabetes patients 86% patients with cvd 87% patients with lung disease 76%

40 European Practice Assessment (EPA): indicators for organization and management of practice (Grol et al, TOPAS Europe 2005) Starting point: framework of domains, dimensions and indicators developed for practice assessment (VIP) in Netherlands (tested in 1600 practices) Set of European indicators for practice organization and management in primary care: national panels in 6 countries (UK, France, Belgium, Switzerland, the Netherlands, Germany) reached consensus about the set of indicators (60 experts)

41 EPA data 270 practices in 9 countries At least one audit last year 48% (11-96) Critical incident register 32% ( 3-89) Patient survey 44% ( 3-85) Recall system diabetics 51% ( 0-100) Recall system patients with cardiovascular diseases 35% ( 0-96)

42 Patients experience EUROPEP: patients evaluate primary care (Grol et al, 2000) Validated international instrument (23 items): patients judging care Data collection in 20 countries Widely used in assessment of primary health care in many countries

43 EUROPEP in 270 practices and more than 8000 patients in 9 countries (Engels et al, 2005) Patients positive about doctor 87% (80-93) Patients positive about organization 80% (67-91)

44 Accreditation system for primary health care in the Netherlands: integration in quality improvement programme Three-yearly (voluntary) full assessment Feedback by visitor: plans for improvement Trained assessors check working on improvement plans and progress made Accreditation based on full cycle of assessment and improvement

45 Quality improvement: finding the balance between watch dog and lap dog

46 Message to clinicians, researchers and policy makers in research We have a serious, underestimated problem in the use of knowledge/evidence in health care Systematic reviewing evidence and development of evidence based recommendations without rigorous efforts and plans for assessment and implementation of results in practice is a waste of health care money Evidence based clinical guideline development should be linked to evidence based assessment and improvement of patient care

47 Conclusions We have a very large and underestimated problem in health care delivery: evidence based guidelines are not followed well in practice Guideline development needs to be more focussed on improvement of patiënt care; we need valid and reliable measurements of actual quality to show room for improvement Guidelines and quality measurements/audits are ideally part of comprehensive systems to improve quality of care Debate about advantages and effects and disadvantages of internal and external measurement of quality Enormous challenges for epidemiologists and health services researchers to bridge gap between guidelines and audit