International Journal for Quality in Health Care 1998; Volume 10, Number 4: pp. 339 343 Local implementation of national guidelines on lower urinary tract symptoms: what do general practitioners in Sydney, Australia suggest will work? MICHELE PUECH 1, JEANETTE WARD 1, GEOF HIRST 2 AND ANN-MAREE HUGHES 1 1 Needs Assessment and Health Outcomes Unit, Central Sydney Area Health Service, Newtown and 2 Taylor Medical Centre, Woolloongabba, Australia Abstract Objectives. Systematic reviews demonstrate that local initiatives are vital to implement nationally developed clinical practice guidelines. Evidence-based guidelines on the management of lower urinary tract symptoms in men were launched by the National Health and Medical Research Council in Sydney in April 1997. A study was conducted through interviews to establish patterns of care in the catchment area before the guidelines were implemented and general practitioners were surveyed in order to ascertain the most useful strategies for local implementation. Design. A four-page questionnaire asked respondents to rate nine items about guideline dissemination; six items relating to the marketing of the guidelines and 15 implementation strategies: conventional educational activities (six); innovative educational strategies (four); quality improvement approaches (two) and patient-based approaches (three). Setting. Sydney, Australia. Study participants. Eighty-three randomly selected general practitioners (50 males; 33 females). Results. Eighty-three out of 108 surveys were returned (77%). Respondents placed high value upon endorsement by eminent individuals and organizations other than the organization developing the guidelines; this was likely to gain their initial attention. One hundred per cent of respondents would be encouraged to use the guidelines if they were promoted as improving quality of care. Implementation strategies preferred by respondents included small group continuing education with a urologist and a general practitioner as a facilitator, lectures and patient education materials. Internet access, interactive computer systems, academic detailing and distance education modules were of least interest. Conclusions. Our method is feasible as a first step in planning local dissemination and implementation for national guidelines. While useful in identifying preferred strategies, its longer-term predictive validity for improving patient outcomes through better guideline implementation needs to be established. Keywords: behaviour change, guidelines, needs assessment, urology Evidence-based clinical practice guidelines have the potential designed to support changes in actual clinical behaviour. Effective to promote more effective health care and improve patient implementation strategies include giving clinicians feedback outcomes [1 3]. However, publication of guidelines alone is about current practice compared with guideline recommendations, insufficient to change clinical behaviour. It is generally accepted paying financial incentives for clinical practice consistent that dissemination and implementation must follow [4]. Dis- with guideline recommendations, withdrawing payments for semination refers to those activities which ensure that a specific practice outside the guidelines as well as providing information, guideline is available to all relevant end-users, encourage posi- checklists or prompts directly to patients [5,6]. Research to tive attitudes towards the guideline and reddress gaps in clinicians evaluate the impact of dissemination and implementation strat- knowledge or skills. Implementation refers to activities egies continues to be recommended [4,5,7,8]. Address correspondence to Prof. Jeanette Ward, Locked Bag 8, Newtown, New South Wales 2042, Australia. Tel: +61 2 9515 3245. Fax: +61 2 9515 3348. E-mail: jward@nah.rpa.cs.nsw.gov.au 339
M. Puech et al. and membership of the group developing the guideline (n= 1) [11,12]. Respondents then were asked to rate each of six items relating to the format and marketing of the guidelines for importance in encouraging them to use the guidelines, using the same five-point scale. Implementation strategies Respondents next were asked to rate the usefulness of each of 15 implementation strategies derived from the international literature [4 6] for usefulness in assisting the adoption of guidelines in clinical practice, again using a five-point scale. Six local educational strategies were listed [lecture; small group meetings; local adaptation with colleagues and practice visits by a peer from general practice or a nurse ( academic detailing ) or a pharmaceutical representative]; four innovative educational strategies (Internet access; interactive computer program; video; distance learning module); two strategies based on a quality improvement approach [feedback on quality of care through the development of a Practice Assess- ment Activity accredited by the Royal Australian College of General Practitioners (RACGP) [13]; and the peer review groups] and three patient-oriented approaches (public cam- paign about the guidelines; written consumer information; a video for men). An open-ended question asked respondents to write any additional comments about guidelines in general or the need for guidelines on urinary tract symptoms in men. Copies of the questionnaire are available upon request. There has also been a growing interest in Australia in clinical practice guidelines to improve outcomes of health care. Through its Quality of Care and Health Outcomes Committee, the National Health and Medical Research Council (NHMRC) initiated the development of evidence-based guidelines in 20 clinical areas in 1995 [9]. A working party was convened through 1996 to develop guidelines about the management of lower urinary tract symptoms in men for general practitioners. A draft of these guidelines was released for comment in August 1996. Endorsed national guidelines were launched in April 1997 [10]. Whereas NHMRC imprimatur lends credibility to clinical guidelines [11], it is not well-placed to organize dissemination and implementation at the local level [9]. Furthermore, very little is known of the effectiveness of implementation strategies in Australian general practice. In response, the working party recommended a randomized trial involving matched localities to compare different implementation approaches, emphasising evaluation of changes in clinical behaviour and, if possible, patient outcomes [10]. To develop the trial protocol, it commissioned a two-part needs assessment [10]. The first part examined patterns of care in general practice by interviewing local general practitioners before the publication of the guidelines (J. Ward, unpublished data). The second part, reported here, required these general practitioners to complete a self-administered questionnaire to identify useful dissemination and implementation strategies. Questionnaire administration and analysis Method Two and 4 weeks after initial mail-out, any non-responder Subject selection and recruitment was telephoned and reminded to return the questionnaire. Participating general practitioners were mailed a thank you A complete list of general practitioners in our catchment area letter at the end of the study. As this was hypothesiswas purchased from a commercial mailing house and a generating research, descriptive statistical analysis only was random sample selected by using computer software. Of the performed, using Epi Info [14]. random sample selected, those who were deceased, retired, uncontactable after six attempts or away from their practice Ethics approval for more than 1 month were considered ineligible. Of 248 The protocol was approved by the Royal Prince Alfred eligible general practitioners, 108 (44%) agreed to participate Hospital Institutional Ethics Committee. in an extensive telephone interview in which two case scenarios were discussed. At the end of this interview, these 108 general practitioners were asked to complete a selfadministered questionnaire to be mailed to them in order to Results identify useful dissemination and implementation strategies. GP sample Questionnaire content Of those 108 local general practitioners who participated in the telephone survey about treatment patterns, 83 (77%) Dissemination strategies participated in this second part. Table 1 lists their demographic The four-page self-administered questionnaire first asked and professional characteristics. There was no evidence of respondents to rate each of nine characteristics of guidelines bias in our sample when these characteristics were compared for its importance in gaining their initial attention, using a with available data for the national reference population [15, five-point scale (extremely, very, somewhat, a little or not at 16]. all important). Items derived from dissemination research were included as follows: the importance of the organization sponsoring the development of the guidelines (n=1 item), Questionnaire findings endorsement by specific professional or government or- Respondents placed high value upon endorsement of guidelines ganizations (n=5), endorsement by a respected urologist (n= as likely to gain their initial attention. Endorsement by 1) or by a respected professor of general practice (n = 1) the Urological Society, respected local urologist, the RACGP 340
Implementation of guidelines Table 1 Respondents demographic and professional char- Themes in their comments included an expectation that acteristics (n=83) guidelines need to be developed by multidisciplinary panels with more than one representative from general practice; that n % Reference sample guidelines should be evidence-based but acknowledge grey (%) 2 zones ; that, in format, guidelines should be precise, brief... and useful as memory joggers ; and that guidelines should Age (years) be used only as an aide and not in litigation. <35 13 16 17.8 35 44 28 35 34.8 45 54 23 28 25.5 55 64 11 14 11.8 Discussion >65 6 7 10.1 Despite an increasing international interest in clinical practice Sex guidelines, it is rare for studies to be conducted in order to Male 50 60 61.9 identify the preferences of the intended target group regarding Female 33 40 38.1 dissemination and implementation strategies. As local support Time commitment is essential if nationally developed guidelines are to change in practice clinical practice [12,17], regional studies using the survey Full time 60 73 76.4 method described here represent a useful preparatory step Part time 22 27 23.6 by which to strengthen local implementation by identifying Membership of local strategies of particular interest and acceptability. We recognize division the potential for non-representativeness of our sample, given Yes 63 76 78.1 its small size and low response rate but balance these method- No 16 19 21.1 ological concerns against our having contacted a larger num- Unsure 2 2 ber of practitioners than has been reported before and obtained their views in a timely manner. RACGP affiliation In this instance, general practitioners initial awareness of Yes 43 52 48.2 guidelines on the management of lower urinary tract symp- No 39 47 50.6 toms in men will be heightened by promoting their clinical 1 usefulness and basis in evidence rather than expert consensus. Where data were missing, columns will not add to 100%. 2 Reference sample from [15,16]. While the imprimateur of the national health authority which RACGP, Royal Australian College of General Practitioners. has sponsored the development of the guidelines was wellreceived, endorsement by other organizations and eminent individuals appears to be essential. Use of guidelines in and a respected general practitioner academic were rated as Australia might be increased by emphasizing their usefulness extremely or very likely by 64 (77%), 59 (71%), 52 (62%) in improving quality of care rather than reducing litigation and 43 (52%) respectively. Other than publication by the or health costs. NHMRC [57 (68%)] all other factors were rated as extremely Our method also revealed interesting preferences among or very likely by fewer than half of the respondents. this target group for implementation strategies. Small group With regard to guideline characteristics likely to encourage meetings with a urologist and facilitator outranked all others. use, all respondents (100%) indicated that if guidelines about The methodological integrity of the evidence-based re- the management of men with lower urinary tract symptoms commendations of the guidelines could be assured by providing were promoted actively as having been designed to improve standardized teaching materials for such small group quality of care, this would encourage use. Evidence-based meetings. While lectures represent conventional fare in guidelines guidelines were rated by 77 (93%) as extremely or very implementation, a consumer version of the guidelines, likely to be used; as were compact guidelines [68 (82%)] and a video for men and a public campaign were rated highly by those providing practical advice to change patient behaviour respondents. In contrast, some implementation strategies [61 (74%)]. Guidelines which reduced the risk of being sued thought to hold great promise received only low ratings in were so rated by 46 (56%). Providing comparative costs of the present study [18]. Academic detailing was not highly equivalent treatment was rated by the fewest respondents as rated. In addition, the low level of interest in Internet extremely or very likely to encourage use [31 (37%)]. access is consistent with the low rate of computer uptake in Table 2 ranks, in decreasing order, respondents ratings of Australian general practice [11]. These latter strategies should 15 potential strategies as extremely or very useful for local be introduced only cautiously. implementation. Small group meetings with a urologist and In conclusion, we recommend this survey method to facilitator from general practice outranked all others. Consumer others involved in the dissemination and implementation of resources ranked highly. Academic detailing, Internet guidelines as a means of ascertaining preferred strategies for publication, interactive computer systems and distance education local activity. Documentation of local activity is recommended packages were of least interest. to add to knowledge of guidelines implementation [19,20]. Ten general practitioners wrote comments about guidelines. Demonstration of better outcomes for men with lower urinary 341
M. Puech et al. Table 2 Strategies rated by respondents as extremely or very useful for local guideline implementation... n % Small group meetings with urologist and GP facilitator 60 72 Lecture 44 53 Video for general practitioners 43 51 Consumer (patient) guide 43 51 Video for patients 43 51 Patients being made aware of guidelines 39 47 Practice assessment activity 1 39 47 Interactive computer program 23 28 Local adaptation of guidelines 22 27 Internet access to experts for queries 19 23 Peer review groups 19 23 Distance learning module 19 23 Peer Visit 15 18 Nurse visit 6 7 Pharmaceutical representative visit 3 4 1 Specific quality improvement initiatives which must be accredited prospectively by the RACGP as a component of its QA&CE Programme. tract symptoms would represent a gold standard with which a systematic review of rigorous evaluations. Lancet 1993; 342: to evaluate the impact of local efforts [17]. Such a study 1317 1322. would establish the predictive validity of a local survey for 6. Davis D, Thomson M, Oxman A, Haynes RB. Changing planning purposes if outcomes improve after the introduction physician performance: a systematic review of the effect of of preferred strategies. CME. J Am Med Assoc 1995; 274: 400 405. 7. Mulrow C. Critical look at guidelines. In Peckham M, Smith R (eds). The Scientific Basis of Health Services. London: BMJ, 1996: Acknowledgements pp. 51 64. 8. Wensing M, Grol R. Single and combined strategies for im- This study was conducted for the NHMRC Working Party plementing changes in primary care: a literature review. Int J to develop Guidelines for the Management of Men with Qual Health Care 1994; 6: 115 132. Lower Urinary Tract Symptoms with funding from the CDH&FS. Drs Kate George, Mike Sladden, Greg Spark, 9. National Health and Medical Research Council. Guidelines for Johnson Tong and Stewart Sloggett provided useful comments the Development and Implementation of Clinical Practice Guidelines. Canberra: AGPS, October 1995. on questionnaire drafts and Ms Lorraine Winchester advised about data management. 10. National Health and Medical Research Council. Clinical Practice Guidelines for the Management of Men with Lower Urinary Tract Symptoms. Canberra: AGPS, November 1996. References 1. Field M, Lohr K. Guidelines for Clinical Practice: From Development to Use. Washington, DC: National Academy Press, 1992. 11. Gupta L, Ward J, Hayward R. Clinical practice guidelines in general practice: a national survey of recall, attitudes and impact. Med J Aust 1997; 166: 69 72. 11. Gupta L, Ward J, Hayward R. Future directions for clinical 2. Grimshaw J, Russell I. Achieving health gain through clinical practice guidelines: needs, lead agencies and potential disguidelines: 1. Developing scientifically valid guidelines. Qual semination strategies identified by Australian general prac- Health Care 1993; 2: 2432 2438. titioners. Aust NZ J Pub Health 1997; 21: 495 499. 3. Hayward RSA, Laupacis A. Initiating, conducting and mainbreast 12. Ward J, Boyages J, Gupta L. Local impact of the NHMRC early taining guidelines development programs. Can Med Assoc J 1993; cancer guidelines: where to from here? Med J Aust 1997; 148: 507 512. 67: 362 365. 4. Grimshaw J, Russell I. Achieving health gain through clinical 13. Royal Australian College of General Practitioners. Quality Asguidelines II: ensuring guidelines change medical practice. Qual surance and Continuing Education Program 1996 8. Sydney: RACGP, Health Care 1994; 3: 45 52. 1996. 5. Grimshaw J, Russell I. Effect of guidelines on medical practice: 14. Epi Info Software Version 6.04b. CDC Atlanta, 1997. 342
Implementation of guidelines 15. Commonwealth Department of Health & Family Services. and effective professional practice. Qual Health Care 1995; 4: General Practice in Australia: 1996. Canberra: AGPS, 1996. 45 47. 16 Ward J, Donnelly N. Rates of RACGP, AMA and division 19. Hirst G. Clinical practice guidelines: to what end? (letter). Med membership in general practice (letter). Med J Aust 1997; 167: J Aust 1997; 167: 288. 107 108. 20. Holt P, Wilson A, Ward J. Clinical Practice Guidelines and Critical Pathways: a Status Report on National and NSW Development and 17. Thompson R, Lavender M, Madhok R. How to ensure that Implementation Activity. North Sydney: NSW Health, 1996: p. 106. guidelines are effective. Br Med J 1995; 311: 237 242. 18. Freemantle N, Grilli R, Grimshaw J, Oxman A. Implementing the findings of medical research: the Cochrane collaboration Accepted for publication 4 April 1998 343