Using clinical data for GME Alvin Rajkomar, MD Research Scientist at Google Brain Hos pitalis t at Univers ity of California, San Francis co
Disclosures
Grew up in Silicon Valley My background Studied computational science (chemistry and physics) Internal Medicine Residency and Hos pital Medicine Fellows hip at UCSF
My first time ward attending 7pm On-Call Resident to intern: Let s run down to the ED for a fresh admit: esophageal bleed Resident to me: I haven t really dealt with a massive GI bleed before
How often do interns not see a common diagnos is? Rajkomar, Alvin, Sumant R. Ranji, and Bradley Sharpe. 2017. Using the Electronic Health Record to Identify Educational Gaps for Internal Medicine Interns. Journal of Graduate Medical Education 9 (1): 109 12
After (simple) analysis of clinical notes: fairly frequently! Analyzed every note written by an intern and used the EHR to discover the patient s diagnos is. Calculated the most common diagnoses (by number of hos pitalization-days ). Identified common diagnoses that interns ended up not seeing.
After a few months of attending I wrote a program to automatically lis t all the patients I had s een One hospitalist Could you do that for me? Another hospitalist Could you do that for our residents?
Interns wanted to track patient outcomes but didn t routinely Narayana, Sirisha, Alvin Rajkomar, James D. Harrison, Victoria Valencia, Gurpreet Dhaliwal, and Sumant R. Ranji. 2017. What Happened to My Patient? An Educational Intervention to Facilitate Postdischarge Patient Follow- Up. Journal of Graduate Medical Education, August. Accreditation Council for Graduate Medical Education. doi:10.4300/jgme-d-16-00846.1.
What would happen if we gave interns a list of their patients on a light rotation?
When given space, interns used their own data to improve their own performance
Could we feedback data live to help on our QI priorities? Residents picked QI priorities, and metrics were reported back periodically Done biweekly by hand (by residents), but trainees often rotated off service, making feedback stale and unactionable Could it be automated? Would it make a difference?
We automated the process... Patel S, Harrison JD, Valencia V, Mourad M, Ranji S, Rajkomar AR. The Feedback Bundle: A Novel Method of Inpatient Audit and Feedback [abstract]. Journal of Hospital Medicine. 2016; 11 (suppl 1). http://www.shmabstracts.com/abstract/the-feedback-bundle-a-novel-method-of-inpatient-audit-and-feedback/. Accessed October 8, 2017.
Rajkom ar, AR; Patel, S; Va lencia, V; Ranji, SR; Harrison, J D; Prasad, P; Mourad, M. THE DATA TRIAL: A RANDOMIZED CONTROLLED TRIAL OF NEXT GENERATION AUDIT AND FEEDBACK [a bstra ct]. Journal of Hospital Medicine. 2017; 12 (suppl 2). http://www.shmabstracts.com/abstract/the-data-trial-a-randomized-controlled-trial-of-next-generation-audit-and-feedback/. Accessed October 8, 2017....and ran an RCT to see if had an effect It improved performance on measures picked by residents But an unexpected finding...
Attribution matters - is this your data? Interns write notes so easy to find patients they directly took care of Residents contributions may not be reflected in the EHR, so we had to merge residency scheduling data Lessons learned Accuracy matters - is this data correct? If data is inaccurate for a single patient, hard to win back trus t of a clinician Utility of analysis matters When given data in a neutral s etting (as a lis t of patients they have seen) trainees were pleased When given as feedback of performance on QI metrics, trainees had very mixed feelings
Arndt, Brian G., John W. Beasley, Michelle D. Watkinson, Jonathan L. Temte, Wen-Jan Tuan, Christine A. Sinsky, and Valerie J. Gilchrist. 2017. Tethered to the EHR: Primary Care Physician Workload Assessment Using EHR Event Log Data and Time-Motion Observations. Annals of Family Medicine 15 (5): 419 26. Daily work will generate new types of data Voice/Conversation data Click logs
New analytical approaches can reveal insights......but they can veer into being invasive Voigt, Rob, Nicholas P. Camp, Vinodkumar Prabhakaran, William L. Hamilton, Rebecca C. Hetey, Camilla M. Griffiths, David J urgens, Dan Jurafsky, and Jennifer L. Eberhardt. 2017. Language from Police Body Camera Footage Shows Racial Disparities in Officer Respect. Proceedings of the National Academy of Sciences of the United States of America 114 (25): 6521 26.
Use data responsibly Speculative: can coaching through AI help improve performance for trainees regardless of resources of training program? Focus on using data collection and feedback for purposes acceptable to patients and trainees Reduce administrative burden of data entry Daily work should be captured and sufficient Teach trainees how to access and analyze their own data Goals for data collection and us e of the future