Continuously Measuring Patient Outcome using Variable Life-Adjusted Displays (VLAD) Mr. Steve GILLETT Ms. Kian WONG Dr. K.H. LEE HAHO Casemix Office Acknowledgements : 1. Queensland Health Department (VLAD for Dummies and other assistance) 2. T McCracken (Cartoons from the internet)
A Simple Request I don t really care about P4P and all these financial incentives. I just want to treat my patients. Can t you just show me how to use these (casemix) data to do a better job in caring for my patients?
.and the answer is VLAD (Variable Life-Adjusted Displays) Markov Chain model plotting the difference between the actual outcome of care against the predicted outcome for sequences of patients. Predicted outcome is based upon logistic regression models against significant explanatory variables. Control limits based upon Markov Chain Monte Carlo Simulation
Concept Design in the Casemix Office (at Steve s White Board)
But. back on the Front Line Don t worry about the technical details (leave them to the technicians). Help the technicians calibrate the tool based upon your clinical knowledge say where the tool is getting it wrong Make use of the tool in your daily work.
How to Monitor Outcome? Traditional Approach Dr. Foster coming to Hong Kong
Statistics Examined and Variation in HA can be simply described by chance. Even if you do detect a significant difference. What do you do examine 300 records to try and understand why?..never went there again!
More Seriously! The traditional approach of comparing quality results between hospitals can be useful Good work is already being done in HA on surgical outcomes. Issues around:- How to identify reasons for variation Focus on often relatively small but statistically significant differences accuracy of the model Might not detect problems that occur intermittently lost in statistical noise Slow to detect and respond
We need a new approach One such approach is VLAD Intuitive to clincians Continuously monitor outcomes over time Identify sequences of patients that do much better or worse than expected identify specific patients for clinical review Establish rules about when records should be reviewed. Recognise that problems can occur in any hospital Focus on identifying the problem and finding a solution Don t focus on comparing hospitals Calibrate the model so that it is doable set the task proportional to the resources.
How a VLAD is made (a simplified hypothetical example) 10 Patients with a specific condition. The chance of good outcome (alive) is 80% (or 0.8) for normal patients chance of bad outcome (death) is therefore 20%. If a patient has a risk factor X, it reduced the chance of living to 60% (or 0.6).
The 10 patients 1. List each patient in order of their admission date. 2. Give an Actual Score of 1 for a good result (live) and 0 for a bad result (die). 3. Calculate the predicted score for each patient 4. Subtract Actual and prediced 5. Add sequentially 6. Plot
Example Patient in date order 1 2 3 4 5 6 7 8 9 10 Risk Factor Present Yes No No No Yes No No No Yes No Outcome Live Die Live Live Die Die Die Live Live Live Actual Score 1.0 0 1.0 1.0 0 0 0 1.0 1.0 1.0 Predicted Score 0.6 0.8 0.8 0.8 0.6 0.8 0.8 0.8 0.6 0.8 Difference 0.4-0.8 0.2 0.2-0.6-0.8-0.8 0.2 0.4 0.2 VLAD Score 0.4-0.4-0.2 0.0-0.6-1.4-2.2-2.0-1.6-1.4 1.0 VLAD Score 0.5 Net Lives Saved/Lost in Excess of Expected 0.0-0.5 1 2 3 4 5 6 7 8 9 10-1.0-1.5-2.0-2.5 Time
A Real Application (In hospital mortality for patients admitted with AMI) Sequence of 500 AMI patients in 8 major hospitals AMI based upon PDx Predicted outcome is based upon a logistic regression model against significant explanatory factors. We use:- Age and Sex groups Major treatments undertaken (Procedural DRGs*) Probability of death measure based upon secondary diagnoses (3M IRDRG category - risk of mortality) Others (to be included as developed). Control limits based upon Markov Chain Monte Carlo Simulation 10,000 iterations of 10,000 sequential events Detect a doubling of the odds ratio Expect 1 false positive signal every 1,200 cases (1 every 2 or 3 years for most hospitals) Remarks: *- DRG04120: IP NON-COMPLEX RESPIRATORY SYSTEM PROCEDURES; DRG05115: IP CARDIAC CATHETERIZATION; DRG05106: IP OTHER CARDIOTHORACIC PROCEDURES; DRG04102: IP LONG TERM MECHANICAL VENTILATION WITHOUT TRACHEOSTOMY; DRG05140: IP PERCUTANEOUS CARDIOVASCULAR PROCEDURES.
Results Summary (1) Most hospitals stayed within the expected range and failed to hit a control limit
Results Summary (2) One hospital hits the upper control limit suggesting a sequence of better than average results
Results Summary (3) Three hospitals hit the lower control limit multiple times over a limited time period suggesting a possible quality of care issue over a relatively short period of time
Comments Clinical staff should review the 20 records of the 20 odd deaths during that period to determine if there was:- an inadequacy of the VLAD calculations refine model; a chance result (false positive) that can be ignored; a problem in the clinical care process that can be avoided in the future.
Summary (1) Imperfect data or lack of precise definitions should:- not be a reason to monitor outcome to the best of our ability not be a reason for doctors clinically reviewing cases with poorer than expected outcomes be a reason to directly avoid sensitive hospital comparisons Should be a reason to limit the extent of review to only where there is a high likelihood of quality of care issues.
Summary (2) VLAD is insufficient in itself to ensure high quality and good patient outcomes VLAD could provide a useful tool to assist the front line by directing doctors towards small numbers of patients for clinical review VLAD can be easily generated for any agreed outcome variables based upon HA current data and standards A certain number of false signals are likely under any form of statistical review. These can be minimized by refining the model based upon review; changing the magnitude of changes (odds ratio) that signal a review
Thank you Contact Details: Steve: stephengillett2@gmail.com Kian: wkk041@ha.org.hk
Supporting slides
Within Range
Hit Upper Limit
Hit Lower Limit