Description and Evaluation of an Educational Intervention on Health Care Costs and Value

Transcription

1 RESEARCH ARTICLE Description and Evaluation of an Educational Intervention on Health Care Costs and Value Jennifer A. Jonas, BSE, BA, a Jeanine C. Ronan, MD, MS, MSEd, a,b Ian Petrie, PhD, c Evan S. Fieldston, MD, MBA, MSHP a,b ABSTRACT OBJECTIVE: There is growing consensus that to ensure that health care dollars are spent efficiently, physicians need more training in how to provide high-value, cost-conscious care. Thus, in fiscal year 2014, The Children s Hospital of Philadelphia piloted a 9-part curriculum on health care costs and value for faculty in the Division of General Pediatrics. This study uses baseline and postintervention surveys to gauge knowledge, perceptions, and views on these issues and to assess the efficacy of the pilot curriculum. METHODS: Faculty completed surveys about their knowledge and perceptions about health care costs and value and their views on the role physicians should play in containing costs and promoting value. Baseline and postintervention responses were compared and analyzed on the basis of how many of the sessions respondents attended. RESULTS: Sixty-two faculty members completed the baseline survey (71% response rate), and 45 faculty members completed the postintervention survey (63% response rate). Reported knowledge of health care costs and value increased significantly in the postintervention survey (P 5.04 and P,.001). Odds of being knowledgeable about costs and value were 2.42 (confidence interval: ) and 6.22 times greater (confidence interval: ), respectively, postintervention. Reported knowledge of health care costs and value increased with number of sessions attended (P 5.01 and P,.001). CONCLUSIONS: The pilot curriculum appeared to successfully introduce physicians to concepts around health care costs and value and initiated important discussions about the role physicians can play in containing costs and promoting value. Additional education, increased cost transparency, and more decision support tools are needed to help physicians translate knowledge into practice. a Department of Pediatrics, Children s Hospital of Philadelphia, Philadelphia, Pennsylvania; b Department of Pediatrics, Perelman School of Medicine, and c Center for Teaching and Learning, University of Pennsylvania, Philadelphia, Pennsylvania DOI: /hpeds Copyright 2016 by the American Academy of Pediatrics Address Correspondence to Evan Fieldston, MD, MBA, MSHP, Children s Hospital of Philadelphia, 34th St and Civic Center Blvd, Philadelphia, PA fieldston@ .chop.edu HOSPITAL PEDIATRICS (ISSN Numbers: Print, ; Online, ). FUNDING: Supported with internal funds from the Children s Hospital of Philadelphia. POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose. FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose. Ms Jonas helped conceptualize and design the study, carried out the analyses, and drafted and revised the manuscript; Drs Ronan, Petrie, and Fieldston conceptualized and designed the study and critically reviewed the manuscript; and all authors approved the final manuscript as submitted. HOSPITAL PEDIATRICS Volume 6, Issue 2, February

2 INTRODUCTION Despite a growing consensus that physician engagement is essential to ensuring that health care dollars are spent efficiently, 1-4 studies have shown that physicians lack the necessary knowledge around health care costs and value to be effective stewards of health care resources and consistently provide high-value care. 5-7 Thus in 2011, the American College of Physicians issued a statement emphasizing an urgent need for physicians to receive training in providing cost-conscious care and decreasing unnecessary care that does not benefit patients. 8 Because there was no organized curriculum geared toward helping pediatricians enhance their competency in providing highvalue, cost-conscious care, the Children s Hospital of Philadelphia (CHOP) began a multiyear project to develop and implement a curriculum for faculty and trainees on these topics. After conducting a needs assessment 9,infiscal year 2014, CHOP piloted a 9-part educational program called Striving for Value in Pediatrics for faculty in the Division of General Pediatrics (mostly general pediatric hospitalists). As part of an ongoing effort to evaluate, improve, and expand educational programming around health care costs and value, this study used baseline and postintervention survey responses to gauge physicians knowledge, perceptions, and views on these issues and to assess the pilot curriculum. Background on the Curriculum The Striving for Value in Pediatrics curriculum was modeled after topic areas from the American College of Physicians Cost-Conscious Care Curriculum 10 that was launched in 2012 for internal medicine trainees. Topics covered in the curriculum included the following: (1) Health Care Value: The Big Picture and Why We Care, (2) Health Care Costs, (3) Health Insurance, (4) Balancing Benefits, Harms and Costs (5); Effective Decision-Making (6); High-Value Screening and Prevention, (7) High-Value Diagnosis, (8) High-Value Prescribing, and (9) Perspectives From the Hospital s Chief Financial Officer (Fig 1). The first few sessions focused on defining key terms and concepts including the value equation (value 5 quality/cost), the 6 domains of quality as defined by the Institute of Medicine, an overview of various types of costs (such as direct, indirect, fixed, variable, marginal, and step costs), and examples illustrating how these concepts relate to health care. The next sessions covered an overview of different types of health insurance, an introduction to the Affordable Care Act, and a preview of emerging value based payment systems as well as the relationships among costs, charges, and reimbursements. Then the focus shifted to an overview of statistical concepts and methods to aid in effective decision-making. Decisionmaking tools and frameworks that were highlighted included an overview of test characteristics, the threshold model, and likelihood ratios. Sessions also included an overview of useful resources to help enforce concepts from the lessons and apply them to screening, diagnosis and prescribing practices on a daily basis. Although each session focused on a distinct set of topics, they each also included a high-level review of important concepts from previous sessions to continually reinforce key concepts. For the final session, the hospital s chieffinancial officer met with division members to answer questions and share his perspective on the future of value-based health care. The sessions included a combination of lectures, interactive case studies, and discussions about how concepts from the lessons relate to daily practice. A general pediatrician with expertise in the topic areas delivered the lecture portions and facilitated the discussions while a general pediatrician with expertise in medical education facilitated the interactive case studies. Sessions were held once a month from October through June during regularly scheduled division meetings, and a recording was sent to all division members after the sessions were completed. All session recordings, as well as an extensive list of relevant articles, tools, and follow-up materials were stored on the hospital s share site. Participants received 1 hour of continuing medical education credits per session attended. Although participation in the program was strongly encouraged, attendance at the sessions or independent completion of recorded modules was completely voluntary. METHODS In September 2013, before the rollout of the curriculum, all 87 faculty members in the Division of General Pediatrics at CHOP (mostly pediatric hospitalists) received a link to a survey hosted in RedCap, a secure online survey tool. The survey included 2 demographic questions, 4 questions asking respondents to rate their knowledge of health care costs, charges, reimbursements, and value on a 5-point Likert scale ranging from very knowledgeable to completely unaware, and 8 questions that were adapted from a survey that was sent to members of the American Medical Association (AMA) in 2012 to gauge physicians perceptions about their role in containing costs and potential barriers to providing cost-conscious care. 11 In July 2014, after all 9 sessions, the 71 current members of the division (lower than the baseline number due to regular turnover as well as some internal reorganization of which faculty were included in General Pediatrics) received a link to a follow-up survey that repeated all questions from the baseline survey, and added 2 new questions asking respondents to report how many of the sessions they attended and how they would rate the program (Fig 2). Sign-in sheets were used to corroborate self-reported attendance. The CHOP Institutional Review Board determined this study to be exempt from ongoing review. To analyze results, we used Microsoft Excel 2010 (Microsoft, Redmond, WA) and Stata 12.1 (StataCorp LP, College Station, TX). Because of membership changes in the division, the groups receiving the baseline survey and the postintervention survey were slightly different. To check that demographic information was comparable between baseline and postintervention respondents, we compared responses for the 2 samples using Pearson s x 2 test. We used Fisher s exact test to compare the proportion of respondents who were in the at least somewhat knowledgeable or the agree categories before and after the intervention, and we calculated odds ratios to 2 JONAS et al

3 FIGURE 1 Learning objectives for curriculum on Striving for Value in Pediatrics. CFO, chief financial officer. determine effect sizes. To compare responses regarding physicians perceptions about their role in containing costs and barriers to providing costconscious care between the AMA member survey and the baseline and postintervention survey of CHOP general pediatricians, we used 2-sample tests of proportions. We also analyzed responses based on the number of sessions respondents reported having attended, and corroborated these results using signin sheet signatures. To test for trends across exposure categories, we used an extension of Wilcoxon s rank-sum test. Results with P,.05 were considered significant. RESULTS Sixty-two faculty members completed the baseline survey (71% response rate), and 45 faculty members completed the postintervention survey (63% response rate). When comparing self-reported attendance to sign-in sheet signatures, 35 (78%) of the data points matched. Of the 10 discrepancies, 8 self-reported having attended more sessions than sign-in sheets indicated and 2 self-reported having attended fewer sessions than sign-in sheets indicated. However, results were similar regardless of how attendance was calculated. Although the following results are based on self-reported attendance, differences between these results and results from attendance sheet data are noted. Demographics A breakdown of the 2 demographic questions for baseline and postintervention respondents is shown in Table 1. The vast majority of respondents (94% in the baseline survey and 84% in the postintervention survey) practiced in an inpatient setting at least part of their time. Although the proportion of inpatient only clinicians was slightly lower in the postintervention group, the difference was not significant (P 5.29). Similarly, although the proportion of new clinicians (0 5 years in practice) was higher in the baseline sample and the proportion of experienced clinicians (.20 years in practice) was higher in the postintervention sample, differences were not significant (P 5.29). Overall Program Evaluation Respondents who reported having attended at least some of the sessions rated the program highly; 30 (71%) rated the program HOSPITAL PEDIATRICS Volume 6, Issue 2, February

4 FIGURE 2 Survey instrument. 4 JONAS et al

5 TABLE 1 Demographics as excellent or outstanding. No respondents who reported attending at least 1 session rated the program as poor. Knowledge Questions Responses to the 4 knowledge questions, divided by number of sessions attended, are shown in Fig 3. In the baseline survey, half of respondents (31) reported being at least somewhat knowledgeable about health care costs, and the other half was minimally knowledgeable or completely unaware. In the postintervention survey, 29 respondents (71%) were at least somewhat knowledgeable, which was a significant increase (P 5.04). The odds of being knowledgeable about health care costs were 2.42 times greater in the postintervention group than in the baseline group (95% confidence interval: ). The increase in knowledge was also significantly associated with the number of sessions attended (P 5.01). The increase in knowledge was even more significant for health care value (P,.001). Although less than half (30, 48%) were at least somewhat knowledgeable at baseline, 85% (35 respondents) were at least somewhat knowledgeable postintervention. The odds of being knowledgeable about health care value were 6.22 times greater in the postintervention group than in the baseline group (95% confidence interval: ). Similarly, attending more sessions was positively associated with knowledge of health care value (P,.001). At baseline, 29 respondents (47%) were at least somewhat knowledgeable about Baseline Postsurvey Practice location Inpatient, (%) 40 (65) 25 (56) Outpatient, (%) 4 (6) 7 (16) Both inpatient and outpatient, (%) 18 (29) 13 (29) Years in practice 0 5, (%) 29 (47) 13 (29) 6 10, (%) 15 (24) 11 (24) 10 15, (%) 8 (13) 9 (20) 16 20, (%) 4 (6) 3 (7),20, (%) 6 (10) 9 (20) Differences in the distribution of responses between baseline and postintervention surveys were not statistically significant. For practice location and for years in practice, P health care charges and reimbursements. Although these percentages increased to 66% (27 respondents) and 61% (25 respondents), respectively, in the postintervention survey, the increases were not significant. However, the positive association between sessions attended and knowledge of health care charges was significant (P 5.02). For health care reimbursements, the association between sessions attended and knowledge just missed statistical significance (P 5.06). On the basis of recorded rather than selfreported attendance, there was a significant increase in knowledge of health care charges (P 5.04), but the analysis of trend across exposure categories just missed statistical significance (P 5.05). Attitudes and Barriers As show in Table 2, at baseline and postintervention, CHOP respondents strongly agreed that cost to society is important in my decisions to use or not to use an intervention, trying to contain costs is the responsibility of every ordering clinician, and physicians should adhere to clinical guidelines that discourage the use of interventions that have a small proven advantage over standard interventions but cost much more. In both surveys, respondents also overwhelmingly disagreed that there is currently too much emphasis on the costs of tests and procedures and that it is unfair to ask clinicians to be costconscious and still keep the welfare of their patients foremost in their minds. Although respondents indicated that they were open to playing a role in enhancing cost-conscious care, less than one-third agreed that they are aware of the costs of the tests and treatments they recommend at baseline, and the percent remained low postintervention. In addition, close to half of respondents agreed that they find the uncertainty involved in patient care disconcerting both at baseline and postintervention. Finally, respondents agreed overwhelmingly, both at baseline and postintervention that decision support tools that show costs would be helpful in my practice. Changes between pre- and postintervention survey responses for CHOP faculty were not statistically significant. A comparison of responses from general pediatricians at CHOP, and members of the AMA, from all specialties, who responded to similar prompts in 2012, 11 is also shown in Table 2. DISCUSSION This study assessed general pediatricians knowledge, perceptions, and attitudes about health care cost and value and the role physicians should play in containing costs and promoting value in health care, before and after rollout of an educational program on Striving for Value in Pediatrics. On the basis of participant feedback and survey responses, the curriculum appeared to successfully introduce participants to concepts around health care costs and value, and initiated important discussions about the role physicians can play in containing costs and promoting value. A large majority of respondents (73%) rated the program as excellent or outstanding, and both formal and informal evaluations of the program were positive. Postintervention surveys showed a significant increase in knowledge about health care costs and value compared with baseline. Postintervention surveys also showed a strong significant association between number of sessions attended and knowledge of health care costs and value. All respondents who attended 7 to 9 sessions and almost all who attended at least 4 sessions reported being at least somewhat knowledgeable about health care value. Similarly, almost all respondents HOSPITAL PEDIATRICS Volume 6, Issue 2, February

6 FIGURE 3 Distribution of responses to the knowledge questions. The increase in knowledge from baseline to postintervention was statistically significant for health care costs (P 5.04) and health care value (P,.001). The association between knowledge and number of sessions attended was also statistically significant for health care costs (P 5.01) and health care value (P,.001). who attended 7 to 9 sessions reported being at least somewhat knowledgeable about health care costs. This suggests that the pilot curriculum was successful in increasing knowledge and awareness about these issues. The lack of similarly significant changes in knowledge of health care charges and reimbursements likely reflects the fact that the aim of the curriculum was to help clinicians become more mindful of high-value care by considering quality and cost, and not to emphasize specifics about hospital charge masters and reimbursement rates. Thus, although these topics were discussed, they were not covered as extensively as the concept of health care value. Although there were no significant changes in responses to the questions aimed at gauging respondents attitudes about their role in promoting health care value and potential barriers to providing cost-conscious care, this was likely because respondents had high rates of agreement, or disagreement, with these statements at baseline. However, respondents did report barriers to being able to practice high-value care, mostly around not being aware of the costs of the tests or treatments they recommend. This is consistent with the findings from previous studies. 6,12 Thus, unless there is a significant movement to increase costtransparency for clinicians, education about the importance of being cost-conscious can only have limited impact. The significant impact that cost transparency can have on physician behaviors was demonstrated in a study that found that physicians who were presented with test fees at the time of order entry ordered 8.59% fewer tests than physicians who did not have access to price information. 13 Another barrier to implementing costconscious behaviors is that many respondents both at baseline and postintervention reported that they find the uncertainty involved in patient care disconcerting. This discomfort may lead clinicians to order more tests than may be necessary. Although dealing with clinical uncertainty and clinical decisionmaking techniques were discussed in 3 of the 9 sessions, more evidence-based standardized clinical pathways and decision support tools (with readily 6 JONAS et al

7 TABLE 2 Comparison of Attitudes and Behaviors About Physicians Role in Promoting Health Care Value Between Pre- and Postintervention Groups and Responses From a 2012 Survey of AMA Members Baseline Responses, CHOP General Pediatricians, % Postintervention, CHOP General Pediatricians, % AMA Members From All Specialties in a 2012 Survey 11,% Cost to society is important in my decisions to use 73 (45) 83 (34) 54 (1307) or not to use an intervention a Physicians should adhere to clinical guidelines that 95 (59) 93 (38) 79 (1930) discourage the use of interventions that have a small proven advantage over standard interventions but cost much more a Trying to contain costs is the responsibility of every 87 (53) 95 (39) 85 (2079) ordering clinician There is currently too much emphasis on the cost 13 (8) 5 (2) 35 (842) of tests and procedures It is unfair to ask clinicians to be cost-conscious 16 (10) 5 (2) 42 (1023) and still keep the welfare of the patient foremost in their minds Decision support tools that show costs would be 92 (57) 93 (38) 70 (1723) helpful in my practice a I am aware of the costs of the tests/treatments I 32 (20) 37 (15) 76 (1863) recommend a I find the uncertainty involved in patient care disconcerting 48 (30) 41 (17) 56 (1370) Reported percentages represent the proportion of respondents that either strongly agreed or moderately agreed with the statement. a AMA member responses are statistically significantly different from CHOP general pediatricians responses, both pre- and postintervention, in pairwise comparisons (P,.05). available information about pre- and posttest probabilities and likelihood ratios, for example) are needed to make implementing those techniques practical in a busy health care delivery setting. The long-term goal of any educational intervention for physicians is to improve clinical competence. In the case of value education, the ultimate goal is to change physicians behaviors in ways that will improve the quality of the care they provide and reduce the provision of unnecessary interventions that cost money but do not benefit patients. According to Miller s triangle, the 4 stages of achieving clinical competence include: (1) acquiring knowledge, (2) knowing how to apply knowledge, (3) demonstrating the ability to apply knowledge through simulation, and (4) integrating knowledge into practice. 14 Because the field of value education is relatively new, this curriculum was an important step in helping physicians precede through the first 2 stages of achieving clinical competence by increasing awareness and basic knowledge about health care costs and value and initiating discussions about how to apply those concepts in practice. However, to help physicians translate increased knowledge and awareness into behavioral changes, hospitals need to address identified barriers to providing cost-conscious care by providing more cost-transparency and decision support tools for use in real time. Although these tools are not yet broadly available (including at our own institution), to ultimately realize large-scale practice change, this preliminary education is essential to equip physicians with the background they need to talk to patients about these issues, to make the best use of new information and tools as they become available, and to prepare physicians for the future of value-based health care. This study has some limitations. First, because of changes in division membership (as a result of regular turnover as well as some internal reorganization of which faculty were included in General Pediatrics), there was a slightly different set of respondents to the baseline and postintervention surveys. Although demographic information for the 2 sets of respondents was not significantly different, there was a higher proportion of outpatient providers and a lower proportion of newer clinicians postintervention. However, there is no evidence indicating that this would skew results in a particular direction because previous studies have indicated that knowledge about health care costs, charges, and reimbursements does not increase with years in practice. 6 Similarly, because of limitations of the data collection tool, we were not able to link baseline responses to postintervention responses. Second, although our survey captured changes in reported knowledge of health care costs and value, it did not capture whether reported knowledge was associated with actual knowledge or changes in behavior. Future studies should evaluate whether physicians are more likely to identify and recommend high-value treatments after participating in an educational intervention. This can be done through the use of case scenarios, simulation exercises, clinical observations, or pre- and postintervention chart reviews. In addition, because the survey used to gauge change in knowledge was not officially validated, it is not clear that all respondents interpreted the options on the Likert scale in the same way. Third, it is possible that physicians who felt strongly about the importance of promoting health HOSPITAL PEDIATRICS Volume 6, Issue 2, February

8 care value were more likely to respond to the survey, so responses may be skewed in that direction. In addition, because certain topics, such as health care charges and reimbursements and dealing with clinical uncertainly were only addressed in certain sessions, postintervention respondents may not have been exposed to the relevant intervention for certain questions. Finally, our exposure analyses are based largely on reported attendance. Although we did attempt to corroborate these responses with sign-in sheet signatures, each session was recorded and available online, and we were not able to track who watched sessions at a later time. In addition, some physicians who signed in to sessions were called away unexpectedly. CONCLUSIONS The 9-part curriculum, Striving for Value in Pediatrics, which was piloted at CHOP in fiscal year 2014, seemed to successfully introduce physicians (mostly general pediatric hospitalists) to concepts around health care costs and promoting value in health care delivery. It also initiated important discussions about the role physicians can and should play in containing costs and enhancing the value of patient care. However, more sessions are needed to reinforce lessons learned and provide more in-depth guidance on how to implement cost-conscious behaviors. To help physicians translate knowledge and awareness into practice, hospitals also need to provide more cost-transparency and decision support tools for use in real time. Future studies can then evaluate how educational initiatives such as the Striving for Value in Pediatrics curriculum, affect physicians behaviors. REFERENCES 1. Fieldston ES. Pediatrics and the dollar sign: charges, costs, and striving towards value. Acad Pediatr. 2012; 12(5): Gower T. Should doctors consider medical costs? The Boston Globe. April 13, Available at: bostonglobe.com/ideas/2014/04/12/ should-doctors-consider-medical-costs/ GPJM1h30qtz6zpfzrxQGoL/story.html. Accessed August 9, Cooke M. Cost consciousness in patient care what is medical education s responsibility? N Engl J Med. 2010; 362(14): Sachdeva R. The need for physician education in health care costs to enhance efficiencies in care delivery. Pediatrics. 2013;131(6): Graham JD, Potyk D, Raimi E. Hospitalists awareness of patient charges associated with inpatient care. J Hosp Medicine. 2010;5(5): Rock TA, Xiao R, Fieldston E. General pediatric attending physicians and residents knowledge of inpatient hospital finances. Pediatrics. 2013; 131(6): Johnson DP, Browning WL, Gay JC, Williams DJ. Pediatric hospitalist perceptions regarding trainees effects on cost and quality of care. Hosp Pediatr. 2015;5(4): Weinberger SE. Providing high-value, cost-conscious care: a critical seventh general competency for physicians. Ann Intern Med. 2011;155(6): Jonas JA, Davies EL, Keddem S, Barg FK, Fieldston ES. Freelisting on costs and value in health care by pediatric attending physicians. Acad Pediatr. 2015;15(4): American College of Physicians. High value care curriculum for educators and residents Available at: acponline.org/curriculum.html. Accessed August 9, Tilburt JC, Wynia MK, Sheeler RD, et al. Views of US physicians about controlling health care costs. JAMA. 2013;310(4): Allan GM, Lexchin J, Wiebe N. Physician awareness of drug cost: a systematic review. PLoS Med. 2007;4(9):e Feldman LS, Shihab HM, Thiemann D, et al. Impact of providing fee data on laboratory test ordering: a controlled clinical trial. JAMA Intern Med. 2013; 173(10): Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(suppl 9):S JONAS et al

9 Description and Evaluation of an Educational Intervention on Health Care Costs and Value Jennifer A. Jonas, Jeanine C. Ronan, Ian Petrie and Evan S. Fieldston Hospital Pediatrics originally published online January 1, 2016; Updated Information & Services Subspecialty Collections Permissions & Licensing Reprints including high resolution figures, can be found at: This article, along with others on similar topics, appears in the following collection(s): Medical Education ucation_sub Teaching/Curriculum Development urriculum_dev_sub Information about reproducing this article in parts (figures, tables) or in its entirety can be found online at: Information about ordering reprints can be found online:

10 Description and Evaluation of an Educational Intervention on Health Care Costs and Value Jennifer A. Jonas, Jeanine C. Ronan, Ian Petrie and Evan S. Fieldston Hospital Pediatrics originally published online January 1, 2016; The online version of this article, along with updated information and services, is located on the World Wide Web at: Hospital Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it has been published continuously since Hospital Pediatrics is owned, published, and trademarked by the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, Copyright 2016 by the American Academy of Pediatrics. All rights reserved. Print ISSN: