Quality Improvement/PBLI in Residency Using Continuity Clinic as the Site- APPD Workshop 10 Mary Kay Kuzma, Raj Donthi and John D Mahan, Nationwide Children s Hospital Columbus, Ohio
ACGME Competency Practice Based Learning and Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and continuously improve patient care based on constant self-evaluation and life-long learning. Learning the process
American Board of Pediatrics To maintain board certification diplomates must participate in an ABP certified (i.e. meets ABP standards for methodological rigor and successful improvement) quality improvement activity for MOC (maintenance of certification). Making useful changes in the system
Group Discussion #1 How are different programs meeting the resident PBLI-QI requirement? Models?
PDSA Cycle of Quality Improvement Act What changes are to be made? Next cycle? Study Complete the analysis of the data Compare data to predictions Summarize what was learned Plan Objective Questions and predictions (why) Plan to carry out the cycle (who, what, where, when) Do Carry out the plan Document problems and unexpected observations Begin analysis of the data Source: www.njha.com/qualityinstitute/pdf/628200432756pm63.ppt
PDSA to Getting QI Projects Done by Residents in a Residency Program P Practical to do. D Deliverable; a time and a place (a rotation) to do it. S Straight-forward process; clear expectations and time-line. A Accountable; faculty assigned to give feedback and/or follow up.
Resident QI Program Planning Our QI Program Triangle (Process): Site Preceptor Project/Format Work Product/Outcome
NCH/Ohio State Pediatric Residency Program Categorical Pediatric Residents: 84 4 continuity/primary care practices 21 residents, 2-3 attendings each. Each resident does a continuity half day per week, and a primary care month rotation each year in that practice.
Site: Why We Chose to do PBLI-QI in a Continuity Clinic/Practice Place where resident spends the greatest cumulative time in residency. Residents are there longitudinally for three years and have a month long rotation in their continuity practice each year (good time to complete a yearly project).
Site: Why We Chose to do PBLI-QI in a Continuity Clinic/Practice Resident Practice Groups - small enough to study, assess and implement change and attempt to determine results (vs larger hospital systems).
Preceptor: Continuity Practice Preceptor as Good Mentor of PBLI-QI Preceptor has an ongoing relationship with the resident. Available for feedback during and at end of project Accountability if not completed Preceptor has interest in improvement of their shared group practice. Preceptor has interest in growth of their mentee resident.
Project/Format: To do a Resident PBLI- QI Project (and get it done ) Pick a rotation where the residents can be assigned to do a project (without huge amounts of nagging and tracking). Have time set aside in that rotation to work on it (not necessary, but helps to get it done).
Project/Format: Defining Our Resident PBLI-QI Projects We decided to do a bottom-up rather than a top-down approach to QI. Resident as active driver of the QI project - something of interest about their practice.
To Paraphrase the Last Few Slides Residents are a bit like toddlers.* They do much better with distinct structure in their lives. They also respond better if given a bit of choice within that structure. (*will it be grapes, or bananas for lunch, guys?) You get more buy-in.
Project/Format: Our PBLI-QI Project Structure in Primary Care Block Month Week 1 Residents in the month-long clinic rotation discuss and define topic they wish to investigate for practice improvement may plan to work as a team. They discuss the topic with their preceptor to receive approval.
Project/Format: Our PBLI-QI Project Structure in Primary Care Block Month Week 2 Define specific patient population. Request list of medical record numbers that match query. Develop a tool to organize data collection.
Project/Format: Our PBLI-QI Project Structure in Primary Care Block Month Week 3 Each resident is given a half day session to review and extract data from the medical records. Week 4 Write up results and propose QI plan on PBLI- QI Final Report form; submit to mentor and the residency program.
Work Product/Outcome: NCH PBLI-QI Report Components What was your question for assessment and improvement? Parameters (who, diagnosis, time frame). Findings in your review. Did you answer your question; if no, why not? Briefly summarize what could be done to improve your practice and how best to implement this change. Data collection tool
Work Product/Outcome PBLI-QI Report Value of electronic document: Send to other residents/attendings in that practice. Disseminate potential changes/improvements for all practices. Place on Residency Intranet site where other residents can use it for future QI. Copy printed out and placed in the resident s portfolio.
Results: Types of Projects Over the last three years there have been 25 to 30 distinct project topics each year. Most projects have been medical record reviews, some have included surveys of the residents/faculty via Survey Monkey or patient s families with paper forms. Many projects are concerned with new guidelines/recommendations and how well we have adopted them in our practices.
Summary of Project Types 2006-2009 Prevention/Safety (AG, screens) Immunizations/ID (immunizations, Synagis, TB) Nutrition (obesity, lipids, Fe def, Vit D) Clinic Operations (flow, schedule, documentation) Asthma/Atopy (staging, action plans,rx) Behavioral Issues (mostly ADHD) Hypertension (recognition/evaluation) 0% 5% 10% 15% 20%
Survey of Residents Experiences and Examples of Resident QI projects
Survey of Residents Experiences SurveyMonkey N = 71 (63% of all the residents who have completed a QI project this year)
Residents feedback on QI Process Your project was acknowledged by your attending and/or other clinic members. (1=none, 5=great) 5 4 3 2 Your project led to changes in your clinic and/or other clinics. (1=no changes, 5=major changes) 1 5 0 5 10 15 20 25 4 3 2 1 0 5 10 15 20
Residents rating of their experience Your overall learning experience from doing one or more QI projects. (1=useless, 5-very helpful) 5 Your work on the QI project(s) changed your curosity/interest in the routines and functions of outpatient clinic work. (1=no increase, 5=I routinely question things I do in clinic) 4 5 3 4 2 1 0 10 20 30 The specific ability of the QI project(s) to prepare you for doing similar work in fellowship or in practice. (1=useless, 5=very helpful) 3 2 1 0 10 20 30 5 4 3 2 1 0 10 20 30 40
Selected QI Project Highlights Vitamin D supplementation Patient Handout utilization ADHD assessment Smoking cessation education/intervention
Vitamin D Question: Are residents evaluating how much vitamin D toddlers are receiving in their diet and prescribing appropriate supplementation based on AAP guidelines? Method: SurveyMonkey to survey residents and faculty EPIC EMR chart review Survey sent ~2 months after new guidelines were published
Vitamin D Survey Results 77 respondents (57 residents, 20 attendings) 59 (77%) aware of the new recommendations 46 (61%) prescribing to nursing infants >81% of the time 59 (76%) prescribing to nursing infants >61% of the time 60 (78%) prescribe to toddlers drinking <32 oz milk/day less than 40% of the time
Vitamin D EMR chart review results
Patient Handouts Question: How often do Residents distribute the handouts to patients/families? Did the self-reported frequency of use improve after intervention? Method: Online survey (yes/no questions) to all of the residents at a Resident Continuity Clinic in April 2009 and January 2010.
Patient Handouts- Intervention After the survey in April 2009, the resident organized a shelf system with pre-printed copies of the handouts used most often. Allowing residents to have a readily-available supply of handouts: no longer waiting on the printers, delaying patient care and disposition. Prepare several new patient handouts on frequently-discussed topics for which there are no Patient/Family handouts.
Patient/Family Educational Handouts- Results 69.0% Percentage of residents April 2009 January 2010 35.7% 57.1% 31.0% 0.0% 7.1% Almost never Sometimes Almost always Frequency of Handout use April 2009: N= 14; January 2010: N=16 100% survey response rate for both time periods.
ADHD Question: Do residents routinely check for the side effects of ADHD medications at medicine evaluation visits? (specifically: hypertension, nausea/vomiting, changes in appetite and sleep, headache, weight loss) Method: Chart Review of all patients 4 to 10 years of age with visits coded with ADD or ADHD (ICD-9 code 314.xx) from January through June 2006 at one Resident Continuity Clinic site
ADHD- results Percentage of visits where side effect (present/absent) was documented 96% 70% 65% 56% 62% 44% Elevated blood pressures Nausea and Vomting Appetite changes Sleep pattern changes Headaches Weight Loss Total visits = 103, Total unique patients = 51
ADHD- Intervention
Second-hand Smoke exposure Question: Is there improvement in resident documentation of second hand smoke exposure after an educational and practice intervention?
Second-hand Smoke exposure Intervention: Attending physicians review a Powerpoint presentation discussing its effects, the importance of a parental discussion, and statistics and resources available to parents. Reinforcing resident documentation of their discussion with parents (specifically adding Second Hand Smoke Exposure to the patient s problem list). Quit Line business cards and handout on smoking cessation resources available for distribution to parents.
Second-hand Smoke exposure Method: EMR Chart review from July-Aug 2009 (preintervention) and Sept-Oct 2009 (post-intervention). Diagnosis codes for well child visits and sick visits for asthma, sinusitis, acute otitis media, and URI. SurveyMonkey of residents asking about selfimprovement in behaviors. Random selection of well and sick visits: 25 pre- and 25 post-intervention.
Second-hand Smoke exposure Resident Survey results Post-intervention improvement in asking parents about smoking since starting project. Pre-intervention documentation of parental smoking discussion occurred in < 50% of visits. Posters are not very helpful. Handouts and Quit Line Cards useful, but difficulty to keep track of them to hand out.
Second-hand Smoke exposure Chart Review results 64% % of charts with documentation Before 8% 40% After 8% Well visits (p < 0.05) Sick visits (p < 0.01)
Successes Diversity Diverse, interesting, comprehensive topics addressed. Uncovered Need for Improvement Projects uncovered many areas with need for improvement. Relevance Residents generally did very good job, were committed to the projects, and interested in improving their practices.
Successes Improved Care In some cases, very good care improvement ideas were generated. Actual QI implementation Some were easily implemented into the practices (some not). Resident QI Projects Completed Residents actually did the projects - therefore we fulfilled the RRC/ACGME requirement while learning broke forth.
Small Group Session #2 Successes In resident PBLI-QI projects that you have done, what have been the successes in your institutions? Resident QI formats/projects that provided good learning Resident QI formats/projects that provided good improvement in care What were good ideas that worked?
Lessons Learned Many of the residents solutions involved changes to the EMR. Not a panacea (surprisingly enough) Many of the solutions to improve care were very time-consuming actions. Can a solution even be done in the patient time and flow available? Frequently the QI priorities of what should be changed were not shared by everyone in the practice.
Challenges to this PBLI-QI Model With several projects each month, how do you implement all the changes, even if there is agreement that they should be implemented? If no change occurs from their project, the residents may feel that there was no real point in doing the project.
Challenges to this PBLI-QI Model Focus frequently is this is what s wrong with our practice ; not addressing how to improve my practice. Generally residents come up with ideas for improvement, leaving the implementation ideas to others (the attendings). Not true QI
Challenges to this PBLI-QI Model Very difficult to complete full PDSA cycle - assess, make changes, re-evaluate - when residents go off to other rotations. One solution - many residents have looked at practice data for the same QI question during their Primary Care Clinic rotation in subsequent years.
Small Group Session #3 Barriers - Solutions Challenge of trying to integrate (or not integrate) with ongoing QI projects being done by the hospital and administration. What barriers did you come across in your implementation of resident PBLI- QI projects in the health care system? What are potential solutions?
Small Group Discussion #4 Future Improvements How to standardize response to QI projects (evaluation piece)? How to challenge residents to take ownership for solutions? Will continual, short PDSA cycles interspersed during training be effective? How to connect to institutional QI system (physician and other staff) to promote system implementation of good proposals?
Questions?