Incorporating feedback and assessment of prescribing performance for junior medical doctors the development of an initiative to improve prescribing practices Danny Soo Clinical Pharmacist, SCGH Nam-Anh Nguyen Medication Safety and Clinical Pharmacist, SCGH WAMSG Symposium 22/09/2015 Delivering a Healthy WA
Introduction Prescription free of error/ambiguity = safe, effective communication of prescriber s intention Large audits can be used as general feedback to clinicians Individual performance assessment and feedback?
Purpose Explore idea of individual assessment and feedback for prescribers to improve prescribing practices Three-stage initiative
Methods 2 intervention audits 1 st audit: Daily audit of the NIMC chart for incidents of errors Daily individual specific and timely feedback (2 weeks) 2 nd audit: Link feedback to assessment Score of 1 to 5 given to prescribers based on error rate (baseline and end-of-term, not daily) Critical errors considered
Methods (2) 3 rd /current phase Assessment +/- feedback at mid-term and end-of-term Same scoring as 2 nd audit but based on pharmacist experience/encounters with prescriber s charts Aim to reduce workload Feedback only required for underperformance Scores to consultant at end-of-term
Error Rate Results 1 st audit n=20 Average error rate decreased from 0.67 to 0.42 errors per prescription (p<0.0001) Error Rate by Date 1.4 1.2 1 0.8 0.6 Error Rate Linear (Error Rate) 0.4 0.2 0 12/05/2014 14/05/2014 16/05/2014 18/05/2014 20/05/2014 22/05/2014 24/05/2014 26/05/2014 28/05/2014 Date 30/05/2014 01/06/2014 03/06/2014 05/06/2014
Results (2) 2 nd audit n=6 Error rates improved from 0.4 to 0.2 errors per prescription and overall scores improved by one point (from 3 to 4) Average error rates improved from 0.53 to 0.41 errors per prescription and average overall scores improved by one point (from 2 to 3) Nil critical errors
Results (3) Overall error rate Total A B C D E F Pre Feedback 0.53 0.9 0.6 0.25 0.4 0.25 0.8 Post Feedback Overall improvement X = Deteriorated = Significant improvement ~ = no significant change 0.41 0.81 0.32 0 0.24 0.3 0.79 ~ ~ ~
Results (4) 3 rd /current phase To determine time taken to assess/feedback Average scores at mid-term compared to endof-term Improvement of scores
Limitations Small samples Prescribing practices only Not assessing clinical appropriateness/indication, omissions Some improvement May further improve with mandatory training and culture of assessment/feedback Phase 3 less audit-intensive method may compromise objectivity Effect/impact of assessment/feedback program on prescribing practices vs. other factors Sustained improvement
Conclusions Individual, timely feedback coupled with assessment has the potential to improve prescribing practices Time/audit-intensive programs may not be sustainable Assessment/feedback program based on simple scoring based on encounters/experience may be a suitable compromise Aim to promote culture of individual accountability and performance
Acknowledgements Clinical pharmacists involved in the audits and prescribing assessment/feedback program Dr Nick Martin, Emergency Department consultant and Head of Post Graduate Medical Education SCGH Medication Safety Committee
Additional slides for Q&A
2 nd audit - method The scoring system is as below: Error rate >0.6 given a score of 1 (rarely) Error rate 0.6-0.4 given a score of 2 Error rate 0.4-0.3 given a score of 3 (Usually) Error rate 0.2-0.1 given a score of 4 Error rate <0.1 given a score of 5 (Always) An error rate of 0.6 (6 errors for every 10 medications prescribed) per prescription is taken from an initial audit which found that the average number of errors per prescription was approximately 0.6 (i.e. error rate above the average is considered unacceptable/underperformance). Additional changes to the score will occur with the following: Not documenting the nature of adverse reaction will downgrade the score by 1 point Not documenting the adverse reaction downgrade by 2 point Charting a medication that the patient has a documented allergy to will be given a score of 1 Unsafe dosage or double prescribing (charting both the generic and brand name of a drug as 2 separate drugs) will be given a score of 1 Any error that is deemed to place the patient at risk or harm will be given a score of 1
Criteria for assessment Junior Doctor Term Assessment Domain 2: Clinical practice The Junior Doctor as practitioner Field 2.7: Prescribe medications safely, effectively and economically, including fluid, electrolytes, blood products and selected inhalational agents. Rating/score descriptor 5: Identifies the appropriate patient medications, fluids, and/or blood product requirements. Consistently prescribes and initiates therapy safely and adheres to all relevant protocols. 4: Performance level between 5 and 3 3: Prescribes safely in most situations and seeks assistance when needed. 2: Performance level between 3 and 1 1: Makes frequent prescribing errors including inappropriate identification of therapy, poor documentation, and disregard for relevant protocols. Does not seek assistance. The score given to the consultant at the end-of-term will be used by the consultant for the above section (2.7) of the Junior Doctor Term Assessment.
Initial score based on error rates: Score Errors per prescription Error rate per 10 prescriptions 1 (rarely) >0.6 >6 2 0.5-0.6 5-6 3 (usually) 0.3-0.4 3-4 4 0.1-0.2 1-2 5 (always) <0.1 <1
Types of errors: Incorrect drug charted Incorrect form of drug charted Incorrect dose/no dose Incorrect frequency/no frequency Illegible/unclear/ ambiguous prescription Unacceptable abbreviation used Prescription order not signed PRN orders without hourly frequency or max dose
Adjust score based on any critical errors: Critical error Not documenting the nature of adverse reaction Not documenting the adverse reaction Charting a medication that the patient has a documented allergy Unsafe dosage or double prescribing (charting both the generic and brand name of a drug as 2 separate drugs/orders) Any error that is deemed to place the patient at risk or harm Adjust score by Downgrade the score by 1 point Downgrade by 2 points Give a score of 1 Give a score of 1 Give a score of 1
Assessment record: Doctor s name 1. 2. 3. 4. 5. 6. Mid-term score (out of 5) (provide feedback via feedback form if score 1 or 2; provide score to consultant at end-of-term only if still underperforming at end-ofterm) End-of-term score (out of 5) (provide this score to the consultant 2-3 weeks before the end of term)