The OAT Analysis Toolkit

August 2012 The OAT Analysis Toolkit A Quality and Patient Safety Roadmap UPMC St. Margaret Hospital

Table of Contents Introduction... 3 OAT Analysis Roadmap... 5 Executive Summary... 7 Top 22 Questions... 11 OAT Survey and Results... 17 Demographics... 17 Safety Culture... 19 Leadership... 21 Measurement... 22 Medication... 26 Purchasing Practice... 28 Procedural... 31 Risk Management... 36 Nursing Practice... 38 Communication... 41 Infection Control... 45 Resources and Toolkits... 49 2

3 Introduction Efforts to transform quality and patient safety in hospitals are sweeping across the country through the Partnership for Patients (PfP) initiative. As part of this initiative, The Hospital & Healthsystem Association of Pennsylvania (HAP) was awarded a federal contract to create the Pennsylvania Hospital Engagement Network (PA-HEN) to promote strategies that foster a culture of safety and reduce adverse events. Those hospitals that joined the PA-HEN were given the opportunity to complete the Organizational Assessment Tool (OAT), a self-assessment tool that was developed by the National Content Developer (NCD), the PfP technical contractor. This self-assessment tool was comprised of 138 questions that addressed ten content domains that affect quality and patient safety. These domains include Safety Culture, Leadership, Measurement, Medication, Purchasing Practice, Procedural, Risk Management, Nursing Practice, Communication, and Infection Control. The OAT was meant to assist hospitals in identifying the use of quality and patient safety evidence-based best practices within their hospitals, as well as new areas of opportunity where suggested practices could be implemented. Many hospitals found that simply completing the OAT helped them to discover gaps in their organizational culture and areas of practice that needed to be re-evaluated. The OAT also can help hospitals identify cultural and leadership factors that affect patient safety and quality initiatives. Evaluating your facility s culture and administrative systems will help hospital leaders determine the facility s readiness for change. In order to assist hospitals in this process, the PA-HEN leadership developed the following toolkit. This toolkit includes a high level aggregate analysis of how you and other hospitals in the PA-HEN responded to the OAT questions, and provides other analytic tools that can be used to identify strong areas of practices as well as opportunities for improvement. In reviewing the average domain scores, there were three domains that represented the highest areas of success and three that represented the greatest opportunity for improvement across the PA-HEN. The three highest scoring domains were Measurement, Nursing Practice and Purchasing Practice. The three lowest scoring domains were Leadership, Safety Culture, and Communication. Not only do these lowest scoring domains represent the greatest areas of opportunity across the PA-HEN, but they are also the keystones to many other facets of hospital operations and patient outcomes. Many of the components within the Leadership, Safety Culture, and Communication domains greatly affect your hospital s patient safety culture, and we are now discovering that patient safety culture affects more than just patient outcomes it also affects patient experiences. There is a growing body of research that demonstrates a link between the AHRQ Hospital Survey on Patient Safety Culture (HSOPS) scores and Hospital Consumer Assessment of Healthcare Providers Survey (HCAHPS) scores. Furthermore, there is a great deal of evidence that demonstrates how important culture can be to the safety and reliability of an organization.

While PA-HEN leadership recognizes the value of the OAT, the tool does have its limitations. Although the contents of the OAT focus on evidence-based best practices, the format of the OAT is more of a self-assessment tool rather than a validated survey. The aggregate data from the OAT results cannot be used for statistical analyses across facilities. However, PA-HEN leadership does believe that reviewing your results and comparing your responses with other PA- HEN hospitals may help to facilitate discussion and identify areas of opportunity for action regarding patient safety and quality at your hospital. Because the OAT was self-reported, responses to OAT questions contain bias and may not accurately depict the patient safety environment of the hospitals. In reviewing the OAT, and when analyzing your hospital s responses with an interdisciplinary team, it is important to ask and discuss the following thought-provoking questions, Does this response accurately represent the practices or environment in our hospital? and If you asked five front-line staff on a unit about X, what would be their response? Answers to these questions will help guide your next steps and your investigation into whether decisions and policies that have been made by leadership have affected patient care as intended. While there are some limitations with the OAT, these limitations do not negate the value that can come from analyzing your OAT results. After moving through the analysis toolkit, hospitals should be able to identify the practices within the domains that may be contributing to a lower overall domain score. Hospitals also should have a full discussion about patient safety culture, communication and leadership engagement within their facility with the use of the Top 22 questions. Not only are these domains important for performance improvement and sustainability, these three domains represent the greatest opportunity for improvement across the PA-HEN. After discussing these factors, hospitals can then use the toolkits that are provided in the Resources and Toolkits section to determine how to proceed and to implement those actions and changes that are necessary to support quality and patient safety transformation. Thank you for participating in the PA-HEN and for helping to transform quality and patient safety across Pennsylvania. All Pennsylvania hospitals and the patients they serve can benefit from the sharing of each other s practices, experiences, and lessons learned, no matter the size of the facility or where they are on their quality and safety journey. Collaboration, networking and shared learning are vital and valuable components for success in achieving the national Partnership for Patients goals, and are fully embodied in the PA-HEN and The Hospital & Healthsystem Association of Pennsylvania philosophy of Achieving More Together. 4

5 OAT Analysis Roadmap Many respondents found that completing the OAT was useful in helping to reflect on quality and patient safety within their facilities. Hospitals responses and domain scores, and overall PA- HEN level scores and response frequencies also should provide a basis for greater review and deeper analysis of quality improvement efforts within facilities. The following steps are meant to generate discussion and give insight as to where efforts should be focused, and how to support PA-HEN initiatives within facilities. 1. Develop and convene an interdisciplinary team to review the OAT. Quality improvement cannot be the sole responsibility of one person or one department. Systemic approaches that include hospital leadership, clinicians, administrators, quality improvement directors, patient safety officers, and front-line staff are most effective in improving and sustaining quality. 2. Review your hospital s domain scores in the Executive Summary on page 7 with your team. The domains that are shown in Exhibits 1 and 2 represent ten areas of focus that affect overall quality and patient safety. Exhibit 1 is meant to give a quick overview of your hospital s highest and lowest scoring domains. To identify possible areas for improvement, hospitals should then compare their domain scores with the average and maximum PA-HEN scores that are provided in Exhibit 1 and Exhibit 2. 3. Closely review the OAT questions that are provided in the Top 22 Questions section. These questions highlight elements of safety culture, leadership, and communication which are vital to the success and sustainability of performance improvement projects. Reviewing these OAT questions will help hospitals identify cultural, structural or administrative obstacles that may affect your PA-HEN projects and other hospital operations. When reviewing your responses to the Top 22 Questions, the following questions may be useful in generating discussions: Does this response accurately represent the practices or environment in our hospital? If you ask five front-line staff on a unit about X, what would their response be? When was the last time that you used policy X, and was it effective in achieving the intent of the policy? Included in the Top 22 Questions section are examples of suggested practices to consider for implementation. Further information about these practices can be found in the Resources and Toolkits section.

4. Drill down into the domains where you have identified opportunities for improvement. Questions from the OAT and the overall PA-HEN level response frequencies are included in the OAT Survey and Results section. Hospital key contacts also have been provided with a PDF copy of their OAT responses as a separate attachment. This attachment can be used as a reference to see how your hospital responded to OAT questions. Responses can then be compared to the frequency of responses by other PA-HEN hospitals. This exercise will help hospitals to determine what items, within their lowscoring domains, that they can improve upon. 5. Discuss your above average domain scores and sustainability plans. Reviewing areas of success are important for identifying what your facility does well and for identifying areas where your quality improvement efforts have been sustained. Explore successful strategies and interventions that can be used for addressing identified opportunities in the low scoring domains. 6. Revisit those questions that ask about the perceptions or practices of front-line staff. Asking front-line staff about culture, best-practices, and patient safety is a good way to investigate whether leadership decisions and organizational policies have affected the patient care that is being delivered. OAT questions that ask about the practices and perceptions of front-line staff include numbers 17, 25, 27, 29, 103, and 116. 7. Use the resources and toolkits to develop a plan of action for identified improvement opportunities. At the end of this toolkit is the Resources and Toolkits section. The resources and toolkits provided can help determine the approaches and tools that would best fit with your hospital in addressing opportunities for improvement. Within this section you can find links to the Implementation Support Packages that were developed by the National Content Developer (NCD). Within these support packages there are guides, articles, and tools that you can use to implement patient safety culture, better communication, and stronger leadership engagement within your facility. To best execute the practices within these support packages, PA-HEN leadership encourages senior leaders to adopt the units where these action plans are being implemented. There also are other resources listed from the Institute for Healthcare Improvement (IHI), information on the Board of Trustee Quality Curriculum that is offered by HAP and information and a link to the PA-HEN PassKey collaborative workspace being utilized to support the PA-HEN initiatives. 6

7 Executive Summary The OAT was created as a self-assessment tool for hospitals to identify areas where they have successfully implemented best practices and to identify areas of opportunity where suggested practices can be implemented. Reviewing and comparing your hospital s scores and responses with overall scores and responses from the PA-HEN is an excellent way to evaluate your successes and areas of opportunity. The exhibits on the following page show a comparison between your hospital s OAT domain scores and the average and maximum PA-HEN domain scores. Exhibit 1 shows a table that displays your hospital s score, and the PA-HEN average and maximum scores for each of the ten domains. The two green domains indicate your two highest scoring domains. The two red domains are your two lowest scoring domains, and may represent your hospital s greatest opportunities for improvement. Exhibit 2 shows a comparison of your domain scores with the PA-HEN average scores using a comparative bar graph. Hospitals are encouraged to review this graph to determine the domains where they scored below the PA-HEN average and above the PA-HEN average. Identifying your above average scores is important in order to identify the practices or strategies that can be used to improve practices in your lower scoring domains. As the PA-HEN leadership team analyzed the overall PA-HEN domain scores and responses, one of the most interesting findings was that the three highest domains, based on average domain scores, were Measurement, Purchasing Practice, and Nursing Practice. The three lowest PA- HEN average domain scores were in Safety Culture, Leadership, and Communication. In creating the Top 22 Questions section that is found in this toolkit, PA-HEN leadership focused on questions from three of the lowest scoring domains, Safety Culture, Leadership and Communication. Not only is it important to focus on these domains because they present the greatest opportunity for improvement, these domains also include elements of culture that contribute to a hospital s overall patient safety culture. Moreover, there is an ever-growing body of evidence that demonstrates a relationship between improvements in patient safety culture, HSOPS and HCAHPS scores. For hospitals looking to improve their HCAHPS scores, in light of pay-for-performance programs, identifying opportunities for improvement in these areas may be one tactic for improving overall patient safety culture, patient care, and patient experiences. Furthermore, these actions and changes also are best implemented when senior leaders play an active role in the operations of a select unit. PA-HEN leadership recommends that senior leaders adopt a unit. When senior leaders adopt a unit, they play a leadership role in the quality improvement projects that are being implemented, and demonstrate to front-line staff that this quality improvement project and these patient safety values are priorities in their facility. If your hospital has scored above the average in any of the lower scoring aggregate PA-HEN domains, you should still analyze areas of opportunity within these three domains in order to

enhance your hospital s patient safety culture. Nevertheless, the exhibits that are included in this section of the toolkit are meant to give hospitals an overview of where they stand in terms of quality and patient safety practices as compared to other hospitals within the PA-HEN. To assist hospitals in identifying and addressing opportunities for improvement within their patient safety culture, PA-HEN, along with the Pennsylvania Patient Safety Authority, will be offering future educational programs. There also will be education on patient safety culture improvement included in the various PA-HEN projects as they continue over the next two years. The PA-HEN plans on re-administering the Top 22 questions that were identified as pertinent cultural questions before the end of the PA-HEN contract so that hospitals can assess their success in implementing best practices and addressing culture within their organization. 8

Normalized OAT Score (%) 9 Exhibit 1. Hospital-level and PA-HEN Domain Scores OAT Domains Hospital Score (%) PA-HEN level Average Score (%) PA-HEN level Maximum Score (%) Safety Culture 50.0 49.1 75.0 Leadership 30.0 47.8 95.0 Measurement 76.1 55.3 93.0 Medication 81.3 56.9 93.8 Purchasing Practice 82.7 68.5 98.1 Procedural 52.1 51.1 88.5 Risk Management 65.7 55.4 97.1 Nursing Practice 93.1 62.0 100.0 Communication 48.4 50.4 75.3 Infection Control 81.3 57.8 92.0 Overall OAT 67.1 55.4 79.4 100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 Exhibit 2. Hospital and PA-HEN Domain Scores Summary Your Hospital PA-HEN OAT Domains Please refer to the Executive Summary for a description of Exhibits 1 and 2.

10 Scoring Methodology The National Content Developer, in partnership with IMPAQ, developed and completed the following scoring methodology for the OAT. The OAT was divided into ten domains that represent areas of importance in quality and patient safety. In order to standardize analyses, a scoring methodology was used for each question, and a score was then assigned to each response. Below is a brief description of how the overall domain scores were generated. 1. Each response was assigned a different weighted score, with the use of evidence-based best practices receiving the highest weighted scores and the absence of evidence-based practices receiving the lowest weighted scores, or a zero. 2. For questions that asked respondents to check all that apply, a score of one was assigned to each practice that was checked, and a score of zero was assigned to those practices that respondents did not check. 3. The individual scores for each question and within each domain were then summed. The sum of response scores were then divided by the total possible number of points available within the domain to create the percentages listed in the Domain Scores section.

11 Top 22 Questions This section includes the top 22 questions from the OAT that the PA-HEN leadership identified as important questions that represent an organization s overall culture. There are many questions that are included from the Safety Culture, Leadership, and Communication domains, as well as a few other questions from the other domains. Hospitals should move through each of the questions and discuss their responses, using the PDF copy of your OAT responses, and identify possible opportunities for improvement and action. Questions, Response Options, and Suggested Practices Frequency of Responses % Indicating Use of Practice Question 17: How strongly do you agree or disagree with the following statement? Front line staff perceive that hospital leadership is committed to delivering necessary resources to achieve national benchmark standards on patient safety (e.g. qualified staff, appropriate medical equipment, information technology infrastructure, educational materials, laboratory)." Strongly Agree 16 13.6% Agree 84 71.2% Neither 10 8.5% Disagree 8 6.8% Strongly Disagree 0 0.0% Suggested Practice: Implement leadership and board rounding; Assign senior leaders to units. Question 18: Which of the following is true about your hospital regarding adverse events? (Check all that apply) There is a structure to triage and analyze adverse events 114 96.6% There is frequent (monthly or more frequent) communication of adverse events to staff (e.g., through newsletter articles, meetings, presentations) 68 57.6% Leadership routinely seeks feedback from staff on reported events and on how to reduce adverse events in the future 80 67.8% Suggested Practice: Develop a process for frequent communication of adverse events to staff. Question 19: How are results from your hospital's survey of patient safety culture used? (Check all that apply) Develop quality improvement interventions 84 71.2% Tied to unit score cards 13 11.0% Tied to the hospital s score card 17 14.4% Compared to national standards or benchmarks 71 60.2% To examine the effectiveness of safety programs 74 62.7% Tracked over time to monitor hospital safety culture 81 68.6% Not applicable 11 9.3% Other 2 1.7%

Questions, Responses, and Suggested Practice Frequency % Use Suggested Practice: Develop a tracking system utilizing national standards or benchmarks; Develop quality improvement interventions and communicate these interventions throughout the facility with the possible development of scorecards or dashboards. Question 24: Which of the following is true about the engagement of patients and their families in your hospital? (Check all that apply) They take part in multidisciplinary rounds 18 15.3% They are partners in monitoring for compliance with safety practices 41 34.7% They actively participate on patient safety committees 59 50.0% They participate in root cause analysis 6 5.1% They sit on the board 32 27.1% Suggested Practice: Consider implementing above practices as appropriate Question 25: If five front-line staff were asked at random, how many do you think would be able to describe a reported incident, good catch, near miss, and/or close call that led to a change (improvements in safety)? 0 1 0.8% 1 22 18.6% 2 35 29.7% 3 39 33.1% 4 16 13.6% 5 5 4.2% Suggested Practice: Develop a process for frequent communication to staff regarding adverse events and changes made due to event reporting and knowledge of near-miss events. Question 27: If five front-line staff were asked at random to describe how patient safety leadership rounds have led to changes that improved safety, how many do you think could give a description? 0 6 5.1% 1 27 22.9% 2 21 17.8% 3 22 18.6% 4 12 10.2% 5 28 23.7% No leadership rounds 2 1.7% Suggested Practice: Implement leadership and board rounding; Assign senior leaders to units; Communicate changes that occur as a result of leadership and board rounding. Question 29: If "yes", How strongly do you think front line staff would agree with the following statement "If any leader or physician violated the policy against disruptive and inappropriate behavior, they would be removed from the hospital." Strongly Agree 0 0.0% Agree 25 21.2% Neither 26 22.0% Disagree 50 42.4% 12

Questions, Responses, and Suggested Practice Frequency % Use Strongly Disagree 16 13.6% Suggested Practice: Implement and consistently apply principles of fair and just culture within your facility. Question 36: Which of the following practices is part of the hospital s quality practices around measurement? (Check all that apply) A dashboard of key results is shared at all levels of the hospital from the board to the front-line staff 92 78.0% The data are always presented in a manner that shows the trend over time 102 86.4% The data broadly cover all settings and include more than mandated core measures 100 84.7% The performance of leadership is tied to key safety and quality measures 74 62.7% A significant subset of the results are shared with the public; data include both positive and negative results 51 43.2% Other 7 5.9% Suggested Practice: Consider implementing above practices as appropriate. Question 41: Which of the following is applicable to your hospital? (Check all that apply) Every unscheduled return to the OR is reviewed for quality of care issues 96 81.4% Every unplanned transfer to a higher level of care (ICU, NICU, level 3 nursery, tertiary care hospital) is reviewed for quality of care issues 57 48.3% Review of unplanned transfer for quality of care issues have led to improvements in the last year 44 37.3% Suggested Practice: Consider implementing above practices as appropriate. Question 53: What measures does your hospital include in its dashboard to track pharmacy performance? (Check all that apply) Adverse drug reactions 97 82.2% Adverse drug events 104 88.1% Automated dispensing system discrepancies 84 71.2% Bar code scanning compliance 50 42.4% After hour medication use 34 28.8% IV mixture competency of nursing staff 19 16.1% Narcotic wasting compliance 71 60.2% SCIP core measure compliance 87 73.7% Overtime and medication error rate 22 18.6% Other 9 7.6% Suggested Practice: Consider implementing the above pharmacy dashboard tracking areas as appropriate. Question 57: Which aspects of your hospital's evaluation of the safety of a device are you aware of? (Check all that apply) Different aspects of the evaluation process (e.g., device failure, user errors, interface design flaws, environmental factors, social factors) 94 79.7% Staff responsible for safety evaluation 72 61.0% Techniques used to evaluate safety 59 50.0% Channels used to communicate evaluation results to management and different units (e.g., purchasing department) 70 59.3% 13

Questions, Responses, and Suggested Practice Frequency % Use Actions taken (e.g., by purchasing department) based on results from safety evaluation of devices 77 65.3% Other 5 4.2% Suggested Practice: Review your facility s device evaluation process and consider implementing above practices as appropriate. Question 63: Is there a formal process for patient safety and risk involvement in equipment analysis and due diligence, selection, and purchasing? Yes 78 66.1% No 39 33.1% No Response 1 0.8% Suggested Practice: Consider utilizing a patient safety checklist when performing these functions; Include patient safety and risk management staff in the process and on committees that make these types of decisions. Question 80: Which of the following is true regarding the process and effect of credentialing, privileging and peer review in your hospital? (Check all that apply) Risk management is included in the peer review process of unexpected outcomes and issues relating to deviations from accepted standards of care and other risk exposures 94 79.7% There is a process in place to support an adequate and objective medical staff and a nursing peer review process that demonstrates effectiveness for improvement 80 67.8% Peer review has resulted in changes in protocol, purchasing, or practices 84 71.2% Suggested Practice: Consider implementing above practices as appropriate. Question 81: What does your hospital do to prevent recurrence of serious safety events? (Check all that apply) Active involvement by the CEO in serious safety event prevention and apprised of effectiveness for prevention 72 61.0% Active involvement by senior leadership in serious safety event prevention and apprised of effectiveness for prevention 108 91.5% A measurement process that ensures RCA/FMEA actions are effective in preventing a repeat occurrence 84 71.2% A system to track implementation and effectiveness of programs to reduce serious safety events 86 72.9% Patient safety culture matrix 42 35.6% A user friendly and "safe" reporting tool for front line staff 101 85.6% Other 1 0.8% Suggested Practice: Consider implementing above practices as appropriate. Question 85: Is there a patient safety incident dashboard for communicating risk management and 'lessons learned' information to senior management, the Board of Directors, and hospital staff? Yes 48 40.7% Yes, but only to senior management and Board of Directors 32 27.1% No 36 30.5% No Response 2 1.7% Suggested Practice: Implement and/or enhance two-way communication regarding risk management activities and lessons learned with all levels of the organization. Question 86: In your hospital s staff performance reviews are staff asked about awareness of hospital quality improvement and patient safety initiatives? 14

Questions, Responses, and Suggested Practice Frequency % Use Yes 44 37.3% No 72 61.0% No Response 2 1.7% Suggested Practice: Increase staff awareness and knowledge of quality and patient safety initiatives and consider incorporating as part of staff performance review process. Question 89: Does the board of your hospital actively participate in risk management, quality management, and patient safety? Yes, directly and routinely 63 53.4% Yes, but through a board risk management subcommittee 36 30.5% No 17 14.4% No Response 2 1.7% Suggested Practice: Educate the board on the importance of quality and their roles and responsibility regarding quality, risk management and patient safety principles and activities. Question 101: Do staff regularly perform comfort rounds to assess and address patient needs for pain relief, toileting, and positioning? Yes, hourly 74 62.7% Yes, every 2 hours 32 27.1% Yes, during shift changes 4 3.4% Only when patients call for assistance 8 6.8% Suggested Practice: Implement frequent and consistent rounding. Question 103: If you asked five front line providers how many do you think would respond I always use the repeat back technique with patients. 0 9 7.6% 1 18 15.3% 2 31 26.3% 3 34 28.8% 4 13 11.0% 5 12 10.2% No Response 1 0.8% Suggested Practice: Implement teach-back techniques, and audit for compliance. Question 105: Which of the following does your hospital do to ensure smooth and safe patient transition upon admission, during shift and unit changes, and/or at discharge? (Check all that apply) Provide patients information about their medical conditions and treatment care plan in a way that is understandable to them 100 84.7% Inform patients and family members of the next steps in their care 99 83.9% Inform patients who the responsible provider of care is during each shift and whom to contact if they have a concern about the safety or quality of care 86 72.9% Create opportunities for patients and family members to address any medical care questions or concerns with their health care providers 94 79.7% Involve patients and family members in decisions about their care at the level of involvement that they choose 98 83.1% 15

Questions, Responses, and Suggested Practice Frequency % Use Use a standard handoff communication system, such as SBAR, and a verification process to ensure that information is both received and understood 77 65.3% Effective communication of patient care to the next provider 94 79.7% Other 5 4.2% Suggested Practice: Consider implementing above practices as appropriate. Question 112: Which of the following is true regarding your hospital? (Check all that apply) There is at least one patient and family advisory council (PFAC) 18 15.3% Patients and their family members serve on key service-based and hospital-wide committees 33 28.0% The minutes of PFAC meetings and their accomplishments are provided to the hospital s governing body and its Board of Trustees 14 11.9% Patient and family care experiences are incorporated into the hospital s planning and day-to-day operation 77 65.3% Suggested Practice: Consider implementing above practices as appropriate. Question 116: If you asked five front line staff on a unit about current blood stream infection rates, how many would know? 0 22 18.6% 1 19 16.1% 2 25 21.2% 3 22 18.6% 4 17 14.4% 5 11 9.3% Not Applicable 2 1.7% Best Practice: Infection rates and other HACs should be regularly communicated on dashboards and/or via other communication mechanisms. 16

17 OAT Survey and Results The OAT Survey and Results section provides the frequency of responses and the percentage of respondents who indicate that they use the practice outlined in the OAT questions. This section can be used for a drill down analysis once hospitals have identified successful domains and domains where there are opportunities for improvement. The ten demographics are split out into their own tables, with the domain name highlighted at the beginning of the tables. Questions and Response Options Frequency of Responses Baseline: Total # of Hospitals Reporting n=118 % of Indicating Use of Practice Demographics Question 4: Region Rural 44 37.3% Urban 35 29.7% Suburban 39 33.1% Question 5: Hospital type (Check all that apply) General 86 72.9% Specialized 12 10.2% Teaching 47 39.8% Academic 12 10.2% Non teaching 20 16.9% For-profit 16 13.6% Safety net 3 2.5% Critical access 6 5.1% Community 67 56.8% Not-for-profit 80 67.8% Other 3 2.5% Question 6: What service does your hospital provide to most of its patients? General medical and surgical 104 88.1% Surgical 0 0.0% Psychiatric care 0 0.0% Tuberculosis and other respiratory diseases care 0 0.0% Cancer care 1 0.8% Obstetrics and gynecology 1 0.8% Orthopedic 1 0.8% Chronic disease care 1 0.8% Acute long-term care 1 0.8%

Questions and Response Options Frequency % Use Pediatrics 1 0.8% Cardiac care 0 0.0% Other 7 5.9% 18 Question 13: Does your hospital use an Electronic Health Records (EHR)? Yes 84 71.2% No 33 28.0% No Response 1 0.8% Question 14: If yes, who is your hospital s main Electronic Health Records (EHR) Vendor? Open Responses N/A N/A Question 15: If "yes", Which of the following do you have in your hospital? (Check all that apply) Computerized provider order entry (CPOE) N/A N/A Automated checks for allergies and drug-drug interactions N/A N/A Patient- and disease-specific reminders N/A N/A Electronic nursing documentation N/A N/A Question 16: If "yes", Have you evaluated whether and how an EHR is making a difference in your facility (e.g. on areas of patient care, business practices, provider productivity, risk management, medical errors, and patient education)? n=84 Yes 35 41.7% No 20 23.8% No, but we plan to evaluate our EHR 29 34.5%

19 Safety Culture Questions and Response Options Frequency % Use Question 17: How strongly do you agree or disagree with the following statement? Front line staff perceive that hospital leadership is committed to delivering necessary resources to achieve national benchmark standards on patient safety (e.g. qualified staff, appropriate medical equipment, information technology infrastructure, educational materials, laboratory)." Strongly Agree 16 13.6% Agree 84 71.2% Neither 10 8.5% Disagree 8 6.8% Strongly Disagree 0 0.0% Question 18: Which of the following is true about your hospital regarding adverse events? (Check all that apply) There is a structure to triage and analyze adverse events 114 96.6% There is frequent (monthly or more frequent) communication of adverse events to staff (e.g., through newsletter articles, meetings, presentations) 68 57.6% Leadership routinely seeks feedback from staff on reported events and on how to reduce adverse events in the future 80 67.8% Question 19: How are results from your hospital's survey of patient safety culture used? (Check all that apply) Develop quality improvement interventions 84 71.2% Tied to unit score cards 13 11.0% Tied to the hospital s score card 17 14.4% Compared to national standards or benchmarks 71 60.2% To examine the effectiveness of safety programs 74 62.7% Tracked over time to monitor hospital safety culture 81 68.6% Not applicable 11 9.3% Other 2 1.7% Question 20: What percent of hospital staff respond to the safety culture survey? More than 90% 1 0.8% 75% - 90% 19 16.1% Less than 75% 89 75.4% No Response 9 7.6% Question 21: What percent of physicians respond to the safety culture survey? More than 90% 0 0.0% 75% - 90% 5 4.2% Less than 75% 101 85.6%

Questions and Response Options Frequency % Use No Response 12 10.2% 20 Question 22: Does your hospital have a system to report serious adverse events? Yes 118 100.0% No 0 0.0% Question 23: If "yes", Are all serious adverse events reported in the system within 24 hours of occurrence? Yes, every event in the past 2 years is reported 70 59.3% No, but incident is logged and filled out later 20 16.9% No, not always 28 23.7% Question 24: Which of the following is true about the engagement of patients and their families in your hospital? (Check all that apply) They take part in multidisciplinary rounds 18 15.3% They are partners in monitoring for compliance with safety practices 41 34.7% They actively participate on patient safety committees 59 50.0% They participate in root cause analysis 6 5.1% They sit on the board 32 27.1% Question 25: If five front-line staff were asked at random, how many do you think would be able to describe a reported incident, good catch, near miss, and/or close call that led to a change (improvements in safety)? 0 1 0.8% 1 22 18.6% 2 35 29.7% 3 39 33.1% 4 16 13.6% 5 5 4.2%

21 Leadership Questions and Response Options Frequency % Use Question 26: Does hospital leadership use a checklist to assess the priority of safety on strategic agendas of senior leadership team, high-level operational meetings, and board meetings? Yes 23 19.5% No 93 78.8% No Response 2 1.7% Question 27: If five front-line staff were asked at random to describe how patient safety leadership rounds have led to changes that improved safety, how many do you think could give a description? 0 6 5.1% 1 27 22.9% 2 21 17.8% 3 22 18.6% 4 12 10.2% 5 28 23.7% No leadership rounds 2 1.7% Question 28: Does your hospital have policies against disruptive and inappropriate behavior by staff? Yes 118 100% No 0 0% Question 29: If "yes", How strongly do you think front line staff would agree with the following statement "If any leader or physician violated the policy against disruptive and inappropriate behavior, they would be removed from the hospital." Strongly Agree 0 0.0% Agree 25 21.2% Neither 26 22.0% Disagree 50 42.4% Strongly Disagree 16 13.6% No Response 1 0.8% Measurement

22 Measurement Questions and Response Options Frequency % Use Question 30: How does your hospital measure improvements in health-care workers' hand-hygiene adherence? (Check all that apply) Periodically monitor and record adherence (as the number of hand-hygiene episodes performed by personnel/number of hand-hygiene opportunities) by ward or by service and provide feedback to personnel regarding their performance 111 94.1% Monitor the volume of alcohol-based hand rub (or detergent used for hand washing or hand antisepsis) used per 1,000 patient-days 30 25.4% Monitor adherence to policies dealing with wearing of artificial nails 62 52.5% When outbreaks of infection occur, assess the adequacy of health care worker hand hygiene 54 45.8% Other 6 5.1% Question 31: Is it possible to identify patients in your hospital with each of the AHRQ Patient Safety Indicators (PSI)? Yes, within 24 hours but with 6-week lag due to billing 36 30.5% Yes, but it would take a couple of weeks to obtain 51 43.2% No feasible way to find this data 23 19.5% No Response 1 0.8% Question 32: Would it be possible to identify patients with an International Normalized Ratio greater than 5 (INR>5)? Yes, this can be done electronically within minutes by the lab 93 78.8% Yes, this can be done electronically with a 1-2 week delay 9 7.6% Yes, this can be obtained with chart review 8 6.8% No feasible way to find this data 5 4.2% Question 33: Would it be possible to identify patients who received the drug Argatroban from the list of patients with INR>5? Yes, this can be done electronically within minutes 73 61.9% Yes, this can be done electronically with a 1-2 week delay 15 12.7% Yes, this can be done with INR>5, but would require chart review to remove Argatroban patients 21 17.8% No feasible way to find this data 5 4.2% No Response 4 3.4% Question 34: Is it possible in the hospital s current system to page or e-mail a caregiver when a patient has a lactate > 4 and is likely a septic shock patient? Yes, this practice or something similar is used now 49 41.5% Yes, but it is only used to identify research study patients 0 0.0% Yes, the system allows for it but is not currently in place 31 26.3%

Questions and Response Options Frequency % Use No feasible way to do this 32 27.1% No Response 6 5.1% Question 35: Is it possible to identify a particular group of patients (e.g. all patients with glucose below 50 ml/dl) based on lab work being done within 12 hours of admission or the location of the patient? Yes, the hospital is able to identify patients location and length of time in a setting and integrate that information with other data 83 70.3% Yes, the hospital is able to find admission time but not location information 4 3.4% No, these data cannot be retrieved from the current system 17 14.4% Other 9 7.6% No Response 5 4.2% 23 Question 36: Which of the following practices is part of the hospital s quality practices around measurement? (Check all that apply) A dashboard of key results is shared at all levels of the hospital from the board to the front-line staff 92 78.0% The data are always presented in a manner that shows the trend over time 102 86.4% The data comprehensively cover all settings and include more than mandated core measures 100 84.7% The performance of leadership is tied to key safety and quality measures 74 62.7% A significant subset of the results are shared with the public; data include both positive and negative results 51 43.2% Other 7 5.9% Question 37: Does your hospital have a linked perinatal database (that connects preconception information, maternal medical conditions and pediatric outcomes)? Yes 13 11.0% No 61 51.7% Not applicable 43 36.4% No Response 1 0.8% Question 38: Is it possible to electronically obtain the timeliness of administration for specific drugs? Yes, it is possible to pull the time of any drug administration in any setting 63 53.4% Yes, this can be pulled in some settings (e.g., ER) or for some special medications 29 24.6% Yes, but information is used only by pharmacy and not by leadership 3 2.5% No, can only be obtained with chart review 22 18.6% No Response 1 0.8%

Questions and Response Options Frequency % Use Question 39: Does your hospital conduct checklist-assisted data collection on pressure ulcer (PU) rates (e.g. % of at-risk patients receiving full PU preventive care) and practices (e.g. frequent monitoring of PU incidence and prevalence, comprehensive skin assessment within 24 hours of admission)? Yes 56 47.5% Yes, but only for PU rates 12 10.2% Yes, but only for PU practices 16 13.6% No 32 27.1% Not applicable 1 0.8% No Response 1 0.8% 24 Question 40: What does your hospital do to identify and minimize the risk of unexpected Emergency Department (ED) return visits? (Check all that apply) Review/identify patients who returned unexpectedly to ED within 48 hours of discharge 69 58.5% Immediate initiation of serious safety event investigations to prevent repeat occurrence (minimally within 24 hours of occurrence) 24 20.3% Collect and evaluate process of care quality measures routinely 55 46.6% Use evaluation results to introduce QI initiatives 45 38.1% Unexpected ED return visits are reviewed by leadership outside ED 26 22.0% Unexpected ED return visits are not monitored 11 9.3% Other 15 12.7% Question 41: Which of the following is applicable to your hospital? (Check all that apply) Every unscheduled return to the OR is reviewed for quality of care issues 96 81.4% Every unplanned transfer to a higher level of care (ICU, NICU, level 3 nursery, tertiary care hospital) is reviewed for quality of care issues 57 48.3% Review of unplanned transfer for quality of care issues have led to improvements in the last year 44 37.3% Question 42: Does your hospital participate in the National Database for Nursing Quality Indicators (NDNQI)? Yes 53 44.9% No 64 54.2% Yes, currently a member but does not report data 1 0.8% Question 43: If "yes", Which hospital acquired condition does your hospital report to the National Database for Nursing Quality Indicators (NDNQI)? (Check all that apply) n=53 Patient falls/injury falls 50 94.3% Pressure ulcers-hospital acquired 47 88.7%

Questions and Response Options Frequency % Use Pressure ulcers-unit acquired 43 81.1% CAUTI 30 56.6% CLABSI 29 54.7% VAP 30 56.6% 25 Question 44: If "yes", How does your hospital use data from NDNQI reports? (Check all that apply) n=53 For quality improvement 49 92.5% To compare unit performance with similar units in peer hospitals 50 94.3% For reporting (e.g., Joint Commission or CMS) 18 34.0%

26 Medication Questions and Response Options Frequency % Use Question 45: Has your hospital completed the ISMP Medication Safety Self-Assessment for Hospitals? Yes, the 2011 self-assessment 38 32.2% Yes, in the last 1-5 years 33 28.0% Yes, but more than 5 years ago 22 18.6% Never 22 18.6% No Response 3 2.5% Question 46: What techniques does your hospital have to prevent harm from high-alert medications? (Check all that apply) Standardized approach to treat patients with similar problems using order sets, preprinted order forms, and clinical protocols 104 88.1% Standardizing concentrations and dose strengths to the minimal few needed 114 96.6% Centralized pharmacist- or nurse-run anticoagulation services 38 32.2% Use of reminders and information about appropriate monitoring parameters in the order sets, protocols, and flow sheets 100 84.7% Protocols for vulnerable populations (e.g., the elderly and pediatric) 42 35.6% Other 10 8.5% Question 47: What error-reduction strategies are in place for administration of insulin products? (Check all that apply) Limiting the variety of insulin products on the formulary 106 89.8% Use of standardized protocols and formats for prescribing insulin 99 83.9% Avoiding the use of abbreviations 111 94.1% U500 insulin doses are prepared only in the pharmacy and are patient-specific 42 35.6% Other 17 14.4% Question 48: For diabetic patients is there a regular monitoring for signs and symptoms of hypoglycemia? Yes, at least four times a day 62 52.5% Yes,1-3 times per day 32 27.1% Yes, but less than once per day 0 0.0% No 17 14.4% No Response 7 5.9% Question 49: Does your hospital have checklist-driven protocols for the safe administration and monitoring of oxytocin for induction and augmentation of labor (e.g. Hospital Corporation of America s Pre-Oxytocin checklist)? Yes 42 35.6% Yes, but checklists are used infrequently 1 0.8%

Questions and Response Options Frequency % Use No 27 22.9% Not applicable 41 34.7% No Response 7 5.9% 27 Question 50: Are smart pump drug libraries developed by a multi-disciplinary team of clinicians (doctors, nurses, pharmacists)? Yes 78 66.1% Yes, but not by multidisciplinary team 5 4.2% Not applicable 34 28.8% No Response 1 0.8% Question 51: Does your hospital have a protocol to address narcotic oversedation? Yes 50 42.4% Yes, but only for patients in a pain management program 11 9.3% No 53 44.9% Not applicable 2 1.7% No Response 2 1.7% Question 52: If "yes", OR "yes, but only patients in a pain management program", Does this protocol include specifications for patient monitoring? n=61 Yes 52 85.2% No 7 11.5% Question 53: What measures does your hospital include in its dashboard to track pharmacy performance? (Check all that apply) Adverse drug reactions 97 82.2% Adverse drug events 104 88.1% Automated dispensing system discrepancies 84 71.2% Bar code scanning compliance 50 42.4% After hour medication use 34 28.8% IV mixture competency of nursing staff 19 16.1% Narcotic wasting compliance 71 60.2% SCIP core measure compliance 87 73.7% Overtime and medication error rate 22 18.6% Other 9 7.6%

28 Purchasing Practice Questions and Response Options Frequency % Use Question 54: Is there a process for identifying the 3,000+ recalls a year and assigning responsibility for addressing when appropriate? Yes, there is a formal process to review every recall 105 89.0% Yes, but only to evaluate high-risk recalls 10 8.5% No, there is not 2 1.7% No Response 1 0.8% Question 55: Is there a procedure to notify patients when a product is recalled? Yes 90 76.3% No 26 22.0% No Response 2 1.7% Question 56: Is there a process for tracking supply shortages affecting care and an active program to reduce these shortages? Yes, and results are shared with nursing units and management regularly 88 74.6% Yes 22 18.6% No 6 5.1% No Response 2 1.7% Question 57: Which aspects of your hospital's evaluation of the safety of a device are you aware of? (Check all that apply) Different aspects of the evaluation process (e.g., device failure, user errors, interface design flaws, environmental factors, social factors) 94 79.7% Staff responsible for safety evaluation 72 61.0% Techniques used to evaluate safety 59 50.0% Channels used to communicate evaluation results to management and different units (e.g., purchasing department) 70 59.3% Actions taken (e.g., by purchasing department) based on results from safety evaluation of devices 77 65.3% Other 5 4.2% Question 58: Does your hospital regularly evaluate already-purchased devices for safety and effectiveness (e.g. for ease of installation and user friendliness) and provide feedback to the purchasing department? Yes, and it has impacted procurement decisions 80 67.8% Yes, but it has not impacted procurement decisions 9 7.6% No 27 22.9% No Response 2 1.7%

Questions and Response Options Frequency % Use Question 59: Which of the following does your hospital have to ensure safe and reliable purchases of medications? (Check all that apply) Engagement by legal and risk management departments to better comprehend the differences between a legal and an illegal operation 26 22.0% Development and communication of a policy for purchasing decisions and documentation of exceptions to the policy 63 53.4% Confirmation of receipt of drug pedigree with all appropriate information 56 47.5% Confirmation of wholesaler, distributor, and supplier licensure with authorities 88 74.6% Keeping records of suspect suppliers 41 34.7% Compare and scrutinize purchases and avoid using drug if there are concerns 72 61.0% Reporting of any suspect suppliers to all appropriate authorities/hospitals (e.g., State Board of Pharmacy and the FDA s MedWatch reporting site) 64 54.2% Not applicable 5 4.2% Other 13 11.0% 29 Question 60: Does your hospital regularly evaluate the performance of purchasing managers and contractors? Yes, through 360 evaluations including front-line managers 19 16.1% Yes, but only by top-level managers 71 60.2% No 21 17.8% Not applicable 5 4.2% No Response 2 1.7% Question 61: How strongly do you agree or disagree with the following statement? Patient safety considerations are incorporated into your hospital's purchasing decision of medical and nonmedical devices. Strongly Agree 52 44.1% Agree 58 49.2% Neither 5 4.2% Disagree 2 1.7% Strongly Disagree 0 0.0% Not applicable 0 0.0% No Response 1 0.8% Question 62: Is your hospital's purchasing department routinely updated about inpatient falls due to accidental/environmental reasons? Yes 59 50.0% No 52 44.1% Not applicable 6 5.1% No Response 1 0.8%

Questions and Response Options Frequency % Use Question 63: Is there a formal process for patient safety and risk involvement in equipment analysis and due diligence, selection, and purchasing? Yes 78 66.1% No 39 33.1% No Response 1 0.8% 30