RESEARCH REPORT DECEMBER 2015 The Health Quality & Safety Commission Surgical Culture Safety Survey Research Report Prepared for Health Quality & Safety Commission Prepared by Ltd.
1 1: Executive Summary... 3 Background and objectives... 3 Research Approach... 3 Research Findings... 3 2: Project Background and Objectives... 6 3: Research Approach... 7 3.1 Research Process... 7 3.2 Survey Design... 8 3.3 Piloting... 9 3.4 Data Analysis... 9 3.5 Research Limitations... 9 4: Research Findings New Zealand... 11 4.1 Summary of Key Findings... 11 4.2 Survey results by key area... 12 Contextual (Readiness)... 12 Interpersonal (Teamwork):... 14 Factor 1: Communication... 14 Factor 2: Coordination... 16 Factor 3: Respect... 17 Factor 4: Assertiveness... 18 Factor 5: Clinical Leadership... 19 Practical (Adherence)... 20 Consequential (Other Items)... 22 Additional Questions... 23 4.3 Differences among Professional Disciplines and Key Demographic Segments... 24 5: New Zealand compared to the US... 25 6: Overview of Qualitative Feedback... 34 7: Demographic and Other Variables... 37 Appendix 1: NZ All Results... 40 Appendix 2: Survey Instrument... 42 Appendix 3: Possible Survey Changes for Consideration... 48
2 FOREWORD Surgical Safety Culture Survey Report Research indicates teamwork and communication failures are at the root of many adverse events that occur during surgery. For this reason the Health Quality & Safety Commission (the Commission) is rolling out a programme, Safe Surgery NZ, aimed at improving surgical teamwork and communication. The Surgical Safety Culture Survey was conducted to provide New Zealand baseline data on attitudes and perceptions of surgical team members before our programme began. We intend to repeat the survey again in 2017 as part of the programme evaluation. Surgical safety checklists, briefings and debriefings improve the quality and safety of surgical procedures, and prevent adverse events. Surgical safety checklists are now routinely used during operations in every district health board; however they have not reached their full potential as a tool to improve teamwork and communication within the surgical team. The culture survey results show that, most of the time, team members work relatively well together. Eighty-eight percent of participants agreed plans for patient care are adapted as needed, and surgeons and anaesthesia providers work together as a well-coordinated team. Eighty-two percent said they are encouraged to report patient safety concerns and 80 percent agreed that decision-making is shared among disciplines in response to conditions or issues that arise during operations. However, there is room for improvement in communication. Only 40 percent agreed surgical team members from different disciplines always discussed patient conditions and the progress of operations. Thirty-one percent did not think surgical team members made sure their comments or instructions were heard. Inattention was an issue during team meetings in the operating room. The Commission is working with every DHB to improve communication and better use of tools such as the checklist, to help surgical teams work together better and share important clinical information more consistently. Over time, we expect to see a reduction in surgical adverse events and an improvement in patient safety. I look forward to reporting back to you on the next survey results and the progress of the Safe Surgery NZ programme. Professor Ian Civil Safe Surgery NZ Programme Expert Advisory Group Chair, Health Quality & Safety Commission
3 1: Executive Summary Background and objectives Since 2012, the Health Quality & Safety Commission (the Commission) has had a goal of reducing perioperative harm caused by adverse events and other errors that take place during the peri-operative period. To date the focus of this programme has been largely to support implementation of the WHO Surgical Safety Checklist (the checklist) and measuring its uptake through the Commission s Quality and Safety Markers. Research suggests that teamwork and communication failure are at the core of nearly half of all medical errors and adverse events. The Commission also recognises that the checklist has not reached its full potential as a tool to improve teamwork and communication within surgical teams. Therefore, over the next two years the Commission is rolling out a package/suite of evidence-based teamwork and communications interventions to District Health Boards (DHBs). As part of the overall monitoring and evaluation of the programme, the Commission wanted to conduct a Surgical Safety Culture Survey (SSCS) in order to provide baseline data on the perceptions and experiences of surgical team members across New Zealand DHBs regarding patient safety and the quality of teamwork in operating theatres. This report presents the results of this work. Research Approach This research was conducted as an online survey with surgical team members in New Zealand DHBs. The survey was a modified version of a SSCS developed by the Harvard School of Public Health, with amendments around language differences only. Permission was given by the Harvard School of Public Health for the Commission to use the survey. Contact was made directly with nominated Safe Surgery Champions across New Zealand DHBs detailing the background to this research and requesting their assistance in this work. All DHBs (aside from Lakes) agreed to take part and all DHBs except South Canterbury sent the survey out. Note however that three surgical team members at another DHB identified Lakes DHB as their main place of work and some other surgical team members identified more than one DHB. Responses from up to N=843 surgical team members have been included in the analysis (but note that not all participants answered every question). A total of N=756 fully completed the survey. Research Findings The Harvard team identified four overarching dimensions as part of their conceptual framework for their study: 1. Contextual (readiness to undertake the initiative) includes experience implementing similar interventions (i.e. the checklist), staff attitudes towards the intervention, belief that it is important for patient safety and the cooperation among multiple disciplines 2. Interpersonal (which recognises the need to foster effective teamwork through communication, coordination, respect, assertiveness and clinical leadership) 3. Practical (adherence) - the extent to which surgical team members adhere to established safety practices in the operating room
4 4. Consequential which measures the perceived impact of the intervention on surgical outcomes as perceived by team members. The following chart shows the average agreement score for each survey dimension/factor, as well as the overall average agreement across all survey dimensions/factors. The lower performing dimensions or factors overall are Practical (adherence) (with an average agreement/positive result of 62%), and Communication and Clinical Leadership (63%). Within these areas, the key issues are around: The current lack of briefings and debriefings The effectiveness of briefings and debriefings that are occurring, and The attitude of the physician during surgery. Consistent with previous research, the findings from this project overall indicate that there are issues to do with teamwork and communication during the peri-operative period. Average Agreement (Positive Result) Overall and by Dimension and Factor % Overall 71 Contextual (readiness) 71 Interpersonal (teamwork): 70 Factor 1: Communication 63 Factor 2: Coordination 72 Factor 3: Respect 74 Factor 4: Assertiveness 76 Factor 5: Clinical Leadership 63 Practical (adherence) 62 Consequential (other items) 81 Additional questions 70 0 10 20 30 40 50 60 70 80 90 100 There was encouraging feedback in relation to aspects of the surgical safety checklist. In most cases the Time Out is being used by every surgical team (84% agreed that this is the case). Most participants (88%) also agreed that plans for patient care are adapted as needed and that surgeons and anaesthesia providers work together as a well coordinated team (81%). In addition most (85%) do not think that
5 surgical team members appear to struggle or are unwilling to ask for help (although we note that 15% still disagreed here which, in a surgical context, is significant). Most say they are encouraged to report patient safety concerns (82%) and most agree (80%) that decision-making is shared among disciplines in response to conditions or issues that arise during operations. Buy-in to the surgical safety checklist appears to be high with 96% agreeing that if they were having an operation they would want one to be used. We note however that despite this, there are some implementation issues evident which need to be addressed and the actual behaviour of some individuals and/or teams does not yet reflect this level of buy-in. Key issues identified with respect to the Contextual dimension were to do with a perception among participants that not all surgical team members agree on the importance of using checklists, 35% of surgical team members do not think the Time Out was straightforward to implement and 38% do not think that surgical team members are open to changes to improve patient safety if it means slowing down. The findings also suggest that Communication is a key area of underperformance. The main issues with respect to the Communication aspect of the Interpersonal dimension were that 31% do not think that surgical team members share information when it becomes available and 31% do not think that surgical team members make sure their comments or instructions are heard. Qualitative feedback suggests that inattention is a key issue during team meetings in the operating room. A further issue here is that briefings and debriefings are not yet consistently taking place across DHBs (only half agreed that briefings and debriefings are common practice). Clinical Leadership has also been identified as an area where improvements will be required although note that the issues to do with Clinical Leadership overlap with the other aspects of the Interpersonal dimension, of Respect and Assertiveness. One of the key issues raised relates to how team members communicate with one another and how that impacts on perceptions of respect within teams and the willingness of team members to voice their opinion. A key factor here relates to the hierarchy within teams there is some suggestion that more senior team members do not communicate in a respectful manner and that this can impact on the confidence of other team members for example, 58% of participants do not think that physicians (note that the qualitative feedback suggests that this is mainly with respect to surgeons) do not maintain a positive tone throughout operations. In addition (as part of Respect), 35% report that potential errors and mistakes are sometimes pointed out with raised voices and (as part of Assertiveness) 33% find it difficult to discuss medical mistakes. The other aspect, which forms part of the Interpersonal dimension is Coordination. On the whole, the Coordination results suggest that in most instances, team members work relatively well together (although there is still room for improvement particularly in terms of physicians and nurses working well together and surgical team members helping one another). There was a very low level of agreement however that surgical team members from different disciplines always discuss patient conditions and the progress of operations (40% agreed that this occurs). As mentioned above, the Practical (adherence) dimension is also one of the lower performing areas. A key issue is that while not all DHBs have implemented briefings and debriefings yet, there are shortfalls evident in terms of the effectiveness of briefings or meetings that are being held 33% of participants disagree that for complex patients or cases, peri-operative briefings always include planning for potential problems, 47% disagree that post-operative debriefings always include a discussion of key concerns for patient recovery or post-op management, and 48% disagree that surgical teams always discuss the operative plan before incision and that equipment or other problems discussed in debriefings are addressed in a timely manner. Other key issues raised as part of this research relate to perceived pressure to progress through the surgical list and the impact of this on patient safety, and communication breakdowns, which can lead to delays in starting surgical procedures.
6 2: Project Background and Objectives The Health Quality & Safety Commission (the Commission) is a stand-alone Crown Entity that has a primary role of assisting private and public providers across the health and disability sector improve service safety and quality. Since 2012 the Commission has had a goal of reducing peri-operative harm caused by adverse events and other errors that take place during the peri-operative period. To date the focus of this programme has been largely to support implementation of the WHO Surgical Safety Checklist (the checklist) and measuring its uptake through the Commission s Quality and Safety Markers. Research suggests that teamwork and communication failure are at the core of nearly half of all medical error and adverse event. The Commission also recognises that the checklist has not reached its full potential as a tool to improve teamwork and communication within surgical teams. Therefore, over the next two years the Commission is rolling out a package/suite of evidence-based teamwork and communication interventions to District Health Boards (DHBs). The package s interventions include: Briefing (at the start of the list of procedures) All three parts of the checklist, modified to be used in a paperless form, as a poster on the operating theatre wall (for each surgery) Debriefing (at the end of the list), and Supporting communication tools such as ISBAR (identify, situation, background, assessment, recommendation), two challenge rule, call-outs and closed loop communication. As part of the overall monitoring and evaluation of the programme, the Commission wanted to conduct a Surgical Culture Safety Survey (SCSS) in order to provide baseline data on the perceptions and experiences of surgical team members across New Zealand DHBs regarding patient safety and the quality of teamwork in operating theatres. The survey tool that was utilised for this research was based on the SCSS developed by the Harvard University School of Public Health. The survey used in the New Zealand context was a close replication of the Harvard survey, with some small wording changes to ensure the language was appropriate for the New Zealand context. This report presents the results of the New Zealand SCSS, as well as data comparing the New Zealand results to the original US findings.
7 3: Research Approach This research was conducted as an online survey. A total of N=971 surgical team members answered some or all of the survey, although not all substantively completed a survey. A total of N=843 were considered to have completed enough questions to contribute to the overall data set. These were people who answered at least some of the core measurement questions. A total of N=756 fully completed the survey. 3.1 Research Process The Commission sent an initial email to nominated Safe Surgery Champions across New Zealand DHBs detailing the background to this research and requesting their assistance in either providing databases of surgical team members to Mobius Research or sending out a pre-programmed survey link directly themselves. At this stage in the process, only Lakes DHB opted not to take part. Contact was then made with Safe Surgery Champions (who were nominated by their DHBs for this role) across all New Zealand DHB s (excluding Lakes DHB, who opted out because of their participation in an earlier Proof of Concept project, which involved several surveys of theatre staff), by a director of Mobius, requesting their assistance in sending the survey to all members of their surgical teams. The survey questions were provided to DHBs in advance, on request. Contact was made initially by telephone to Safe Surgery Champions, followed by an email, detailing the process. Further telephone and email contact was made - to update people on the process and also to follow up once the survey and email link had been sent. A prize draw was offered for all surgical team members taking part, as an incentive to encourage a high response rate. All DHBs (aside from Lakes) agreed to take part and all DHBs except South Canterbury sent the survey out. Note however that three surgical team members at another DHB identified Lakes DHB as their main place of work and some other surgical team members identified more than one DHB. Not all Safe Surgery Champions sent the survey to all surgical team members (some sent to selected team members only) and across the participating DHBs sample sizes varied from N=1 (MidCentral) and up to N=169 (Waikato). Not all participants nominated a DHB.
8 Participating DHBs and Sample Sizes The results presented here include some partially completed surveys where the amount of the survey completed was considered substantial enough for responses to be included. DHB N % of total sample Northland 2 0.3% Auckland 111 14.2% Waitemata 110 14.1% Counties Manukau 100 12.8% Waikato 169 21.6% Bay of Plenty 50 6.4% Lakes 3 0.4% Tairawhiti 10 1.3% Hawke s Bay 5 0.6% Taranaki 30 3.8% Whanganui 5 0.6% MidCentral 1 0.1% Capital & Coast 9 1.2% Wairarapa 20 2.6% Nelson Marlborough 12 1.5% West Coast 6 0.8% Canterbury 65 8.3% Southern 73 9.3% Total (where a DHB was nominated)* 781 100% * A further N=190 did not nominate a DHB and/or did not complete enough of the survey 3.2 Survey Design The survey was a close replication of the SCSS developed by Sara Singer and colleagues at the Harvard School of Public Health. 1 Some small wording changes were made for the New Zealand context. All other questions (other than small wording changes) remained the same as in the Harvard survey. The order of the questions was changed slightly in order to ensure a more logical flow for participants. The questions were therefore organised into the following sections: Surgical services worked in/responsibility for checklist implementation Patient safety and surgical safety checklist implementation Communication Teamwork Patient care Planning, briefings and debriefings Demographics/participant background. There was one open-ended question for any additional comments or feedback. 1 Original survey available at http://www.safesurgery2015.org/uploads/1/0/9/0/1010835/2p - surgical_safety_culture_ survey_2011_0603_final.pdf
9 3.3 Piloting The survey wording and structure was piloted with a small number of surgical team members and no specific changes were identified. There was however some (minor) feedback and suggestions from participants taking part in the wider survey, which are discussed in Section 2.6 below. 3.4 Data Analysis While the survey instrument was structured into the sections identified above, analysis of the core measurement questions was based around the analysis conducted by the Harvard team 2. For analysis purposes, the Harvard team grouped the measurement questions as follows. These four overarching dimensions were deemed by the Harvard team to be of greatest interest in the surgical environment and more feasible to obtain by other forms of data collection. 1. Contextual (readiness) 2. Interpersonal (teamwork) a. Factor 1: Communication b. Factor 2: Coordination c. Factor 3: Respect d. Factor 4: Assertiveness e. Factor 5: Clinical Leadership 3. Practical (adherence) 4. Consequential (other items). There were four additional questions included in both the Harvard and the New Zealand survey, which were not analysed as part of these four dimensions. The results of these are presented separately in this report. Results were analysed (by these sections) overall and then also presented by DHB. A cross-tabular analysis was conducted by gender, ethnicity, age, primary role of participant and the number of years working in this role at any hospital. Statistical testing was completed on a wide range of segmented groups within the overall data. The mean for each segmented group was compared to the overall mean for each question. This was to understand if there were any significant differences between the groups within a category compared to the overall sample. The statistical testing of the two means was completed by comparing the means at the 95% confidence level, for a 50% response distribution. 3.5 Research Limitations Results should be viewed in the context of study limitations. Firstly, Mobius Research did not have any control over to whom (specifically, the number of surgical team members) the survey was sent. While every attempt was made to ensure that Safe Surgery Champions understood the need to disseminate the survey, and emailed the survey link to all surgical team members, not all did this. In addition, because we did not hold the database information (other than for Tairawhiti DHB) we were unable to identify via our survey software who had and had not completed a survey. Typically, if we hold the database information and send surveys out directly, we are able to identify people who have not yet started a survey or have started but not completed a survey, and send reminders out to those people specifically. For this survey we could only ask that Safe Surgery Champions send out reminders on our behalf (although these would 2 Surgical Team Member Assessment of the Safety of Surgery Practice in 38 South Carolina Hospitals, Medical Care and Research Review (2015), Sara J Singer et al
10 be non-targeted i.e. people who had completed a survey would also receive a reminder). This approach had implications for the total sample size achieved, although we consider the overall sample size that was achieved of N=971 to be robust. The main implication therefore was small sample sizes for some DHBs. Note also that N=190 participants did not nominate a DHB and that any results presented by DHB therefore excludes these people. Because not all DHBs representatives sent surveys to all surgical team members, comparisons across DHBs are limited and while we can consider trends in results across DHBs, any differences in results by DHB cannot be considered to be statistically significant. In addition, because we did not control administration of the databases, we cannot comment on any non-response bias by individual questions (as was done in the analysis conducted for the Harvard survey) i.e. we do not know what the survey response rate was because we do not know how many surveys were sent out. A further limitation is that some surgical team members, who may be less proactively engaged than other surgical team members with the checklist, may also have been less interested in and less likely to have completed a survey. This may mean that the survey results are more positively skewed in terms of the views and attitudes expressed. Furthermore, survey culture surveys in general tend to skew more positively. 3 3 As with most safety climate surveys (Sexton et. al., 2006; Singer et. al. 2009; Sorra & Nieva, 2012), responses were predominantly positive
11 4: Research Findings New Zealand 4.1 Summary of Key Findings Results in the tables in Section 4 have been presented in terms of most positive (80%+ agreement) to most negative. In order to highlight key possible priority areas moving forward, results have been segmented into percentiles 80%+, 70-79%, 60-69%, 50-59% and under 50%. Any result below 70% has been flagged as a potential priority. Most questions in the survey were asked in the positive (therefore a high agreement score is a positive score). However, some questions were asked in the negative (where an agree score would be considered a negative response). Where this occurs, the results from these questions have been reversed so that in the tables below, results are still able to be presented in order of positivity. The following table shows the average agreement score for each survey dimension/factor, as well as the overall average agreement across all survey dimensions/factors. The lower performing dimensions or factors are Practical (adherence) (with an average agreement/positive result of 62%), and Communication and Clinical Leadership (63%). Within these areas, the key issues are around: The current lack of briefings and debriefings The effectiveness of briefings and debriefings that are occurring, and The attitude of the physician during surgery. Survey dimensions/factors Average agreement score* Overall 71% Contextual (readiness) 71% Interpersonal (teamwork): 70% Factor 1: Communication 63% Factor 2: Coordination 72% Factor 3: Respect 74% Factor 4: Assertiveness 76% Factor 5: Clinical Leadership 63% Practical (adherence) 62% Consequential (other items) 81% Additional questions 70% * Reflects the positive, all negatively worded responses have been reversed for this calculation
12 4.2 Survey results by key area Contextual (Readiness) The first overarching dimension identified by the Harvard team is the Contextual (readiness) dimension. The Contextual dimension asks how ready the surgical team is for undertaking the initiative. Readiness includes but is not limited to, experience implementing similar interventions (i.e. the checklist), staff attitudes towards the intervention, belief that it is important for patient safety and the cooperation among multiple disciplines. These results suggest there is a need to ensure wider buy-in among surgical team members regarding the importance of using checklists. Just under one-third of surgical team members taking part in this survey do not think that all surgical team members agree that checklists are important. This is particularly significant in the context that 96% of surgical team members taking part in this survey agree that if they were having an operation they would want a surgical checklist to be used. In part, views regarding other team members attitude towards checklists may be based around the qualitative feedback that some surgical team members are often inattentive during briefings, and checklists. Therefore, while 96% agree that they would want a checklist as a patient, the behaviour of some when the checklist is being applied does not appear to reflect this. It will be important to consider the reasons behind this behaviour deficit i.e. too many people in the room, not enough time made available, a lack of perceived relevance to certain team members etc. It would also appear that some surgical team members had difficulty implementing the Time Out. It will be important to review the implementation process and why some teams found this process difficult. It may be useful to investigate some best practice examples i.e. why some teams found the process straightforward and what the differences in their approach were. It will also be important to consider the wider issues that may be creating a reluctance to slow down processes even if it means improved patient safety. 80% + 84% agree that the Time Out is used in every case by every surgical team 70-79% 76% do not think that interest in checklist implementation is limited to one profession 60-69% (priorities) 69% agree that surgical team members all agree on the importance of using checklists during surgery 31% disagree 65% did not think the Time Out was difficult to implement 35% did think it was difficult 62% agree that surgical team members are open to changes to improve patient safety even if it means slowing down 38% disagree 50-59% None Under 50% None
13 Contextual (Readiness) 2015 Interest in checklist implementation is limited to one profession e.g. surgery, anaesthesia or nursing (n=838) 76 18 7 Surgical team members all agree on the importance of using checklists in surgery (n=821) 31 42 27 The 'Time Out' was difficult to implement (n=835) 65 30 5 The 'Time Out' is used in every case by every surgical team (n=837) 16 35 49 Surgical team members are open to changes that improve patient safety, even if it means slowing down (n=843) 38 43 19 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
14 Interpersonal (Teamwork): The second dimension is the interpersonal dimension. The interpersonal dimension recognises that implementing interventions, such as surgical safety checklists, requires complex social and behaviour changes that challenge the status quo. Surgical checklists encourage non-hierarchical, team-based interaction, enhanced communication, anticipation of potential complications, and the means for responding to them. In short, they aim to foster effective teamwork. Building on previous models of teamwork the Harvard team defined five measurable interpersonal factors that may affect and be affected by surgical interventions: communication, coordination, respect, assertiveness, and clinical leadership. Communication refers to how well team members share information and listen; coordination addresses how well they work together; respect refers to whether team members feel valued and appreciated; assertiveness addresses the extent to which it is easy for team members to speak up, ask for help, or discuss mistakes; and clinical leadership asks whether the physicians on the team - surgeons and anaesthetists - are seen as good leaders. Factor 1: Communication The findings suggest that communication is a key area of underperformance. Issues identified here suggest that there needs to be better processes in place, which enable the sharing of relevant and important information (and in a timely manner). There needs to be further investigation regarding why this sometimes doesn t occur. Once again, considering some best practice examples across DHBs may be a worthwhile exercise. A significant issue identified here is that not all DHBs have consistently implemented briefings and debriefings. 80% + None 70-79% None 60-69% (priorities) 69% agree that surgical team members share information when it becomes available 31% disagree 69% agree that surgical team members make sure their comments or instructions are heard 31% disagree 50-59% (priorities) 51% agree that team discussions (briefings and debriefings) are common 49% disagree Under 50% None
15 Factor 1: Communication 2015 Surgical team members share key information as it becomes available (n=782) 31 53 16 Surgical team members make sure their comments or instructions are heard (n=789) 31 55 14 Team discussions (e.g. briefings or debriefings) are common (n=792) 49 35 16 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
16 Factor 2: Coordination On the whole, the coordination results suggest that in most instances, team members work relatively well together (although there is still room for improvement particularly in terms of physicians and nurses working well together and surgical team members helping one another). The level of agreement that surgical team members from different disciplines always discuss patient conditions and the progress of operations was very low. 80% + 88% agree that plans for patient care are adapted as needed 81% agree that surgeons and anaesthesia providers work together as a well coordinated team 70-79% 77% agree that physicians and nurses work together as a well coordinated team 75% agree that surgical team members appear eager to help one another 60-69% None 50-59% None Under 50% (priorities) 40% agree that surgical team members from different disciplines always discuss patient conditions and the progress of operations Factor 2: Coordination 2015 Plans for patient care are adapted as needed (n=756) 12 63 25 Surgical team members from different disciplines always discuss patients' conditions and the progress of operations (n=784) 60 34 6 Surgeons and anaesthesia providers work together as a well-coordinated team (n=783) 19 63 18 Physicians and nurses work together as a wellcoordinated team (n=783) 23 61 16 Surgical team members appear eager to help one another (n=782) 25 58 17 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
17 Factor 3: Respect There are some issues evident in the way in which surgical team members communicate with one another which seem likely to impact on the way in which they function as teams. Key issues here are to do with the way in which team members feel they are valued by other team members and the level of confidence team members have in providing input/contributing to discussions. There are clearly some team culture issues, which are impacting here; this can be seen in the relatively high proportion (35%) that report that potential errors or mistakes are sometimes pointed out with raised voices or condescending remarks. 80% + None 70-79% 79% agree that their input about patient care is well received by other surgical team members 77% agree that they are always treated as a valuable member of the surgical team 77% agree that surgical team members communicate with them in a respectful manner 60-69% (priorities) 65% agree that potential errors or mistakes are pointed out without raised voices or condescending remarks 35% disagree 50-59% None Under 50% None Factor 3: Respect 2015 Potential errors or mistakes are pointed out without raised voices or condescending remarks (n=781) 36 53 12 I am always treated as a valuable member of the surgical team (n=776) 23 57 20 My input about patient care is well received by other surgical team members (n=785) 21 60 19 Surgical team members communicate with me in a respectful manner (n=784) 23 58 19 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
18 Factor 4: Assertiveness Team culture is also an issue here. Further investigation into who and what drives team culture will be important, and particularly (once again) looking at teams that are functioning well and why, in order to better understand why there are problems with the way other teams are functioning. Anecdotally (via the qualitative feedback) there appears to be some issues to do with some senior team members (surgeons) and the way in which they are communicating with other team members. 80% + 85% do not think that surgical team members appear to struggle or that they do not want to ask for help 70-79% 77% do not find it difficult to speak up when they perceive problems with patient care 60-69% (priorities) 67% do not find it difficult to discuss medical mistakes 33% disagree 50-59% None Under 50% None Factor 4: Assertiveness 2015 It is difficult to speak up when I perceive problems with patient care (n=783) 77 19 4 Surgical team members appear to struggle and do not ask one another for help (n=779) 85 13 1 It is difficult to discuss medical mistakes (n=773) 67 27 6 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
19 Factor 5: Clinical Leadership Many of the issues overlap (between the areas of Respect, Assertiveness and Clinical Leadership) and continue to in regard to the culture of individual teams and the way in which team members communicate and therefore are able to function effectively as teams. 80% + None 70-79% 79% do not think that physicians are only open to suggestions from other physicians 60-69% (priorities) 68% agree that physicians are present and actively participating in patient care prior to skin incision 50-59% None Under 50% (priorities) 42% agree that the physician maintains a positive tone throughout operations 58% disagree Factor 5: Clinical Leadership 2015 Physicians maintain a positive tone throughout operations (n=782) 58 37 5 Physicians are present and actively participating in patient care prior to skin incision (n=749) 33 47 21 Physicians are only open to suggestions from other physicians (n=782) 79 18 4 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
20 Practical (Adherence) The third dimension identified by the Harvard team is the practical dimension; it refers to the extent to which surgical team members adhere to established safety practices in the operating room (e.g., preoperative planning for potential problems and post-operative debriefing on key concerns for patient recovery and management). This is one of the lower performing areas with the key issue being around the fact that not all DHBs have implemented briefings and debriefings yet, and (based on the results below, for which a nonapplicable option was provided) the effectiveness of briefings or meetings that are being held. The extent and quality of team meetings (including briefings and debriefings) is a key area requiring significant further work. This is particularly important in the context of other measures regarding the perceived ability of team members to speak out about any concerns they have, which may be impacting on the results below (i.e. the ability to discuss key concerns for patient recovery or post-operative management). 80% + 88% of surgical team members refer to each other by their name, not their role 70-79% None 60-69% (priorities) 67% agree that for complex patients or cases, peri-operative briefings always include planning for potential problems 50-59% (priorities) 53% agree that post-operative debriefings always include a discussion of key concerns for patient recovery or post-op management 47% disagree 52% agree that equipment issues or other problems discussed in post-op briefings are addressed in a timely manner 48% disagree 52% agree that surgical teams always discuss the operative plan before incision 48% disagree Under 50% (priorities) None
21 Practical (Adherence) 2015 Equipment issues or other problems discussed in post-op debriefings are addressed in a timely manner (n=703) 48 38 14 Postoperative debriefings always include a discussion of key concerns for patient recovery and post-op management (n=711) 47 37 16 For complex patients or cases, preoperative briefings always include planning for potential problems (n=730) 34 41 26 Surgical teams always discuss the operative plan (i.e. more than the location of the incision and name of the procedure) before incision (n=758) 49 37 15 Surgical team members refer to each other by role instead of name e.g. "Nurse" instead of "Anna" (n=784) 88 9 3 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
22 Consequential (Other Items) The final dimension identified by the Harvard team is the consequential dimension. The consequential dimension measures perceived impact of the intervention on surgical outcomes as perceived by team members (e.g. how safe team members would feel being treated as patients in their own operating rooms). Once again, the wider issues contributing to a reluctance to slow down processes may require further consideration. 80% + 96% agree that if they were having an operation they would want a surgical safety checklist to be used 86% would feel safe being treated as a patient 70-79% None 60-69% (priorities) 62% do not feel that pressure to move from case to case gets in the way of patient safety 38% disagree 50-59% None Under 50% None Consequential (Other Items) 2015 Pressure to move quickly from case to case gets in the way of patient safety (n=838) 62 28 10 If I were having an operation, I would want a surgical safety checklist to be used (n=843) 4 15 81 I would feel safe being treated as a patient (n=833) 14 54 32 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
23 Additional Questions These four additional questions were part of the Harvard study but were not analysed by the Harvard team as part of the four overarching dimensions. It is clear from the results across many areas of this research, that communication within surgical teams is a key weakness that needs to be addressed. 80% + 82% agree that they are encouraged to report any patient safety concerns they may have 80% agree that decision-making is shared among disciplines in response to changes in patients conditions or issues that arise during operations 70-79% 71% agree that disagreements are resolved with an emphasis not on who is right but what is right for the patient 60-69% None 50-59% None Under 50% (priorities) 46% do not think that communication breakdowns frequently lead to delays in starting surgical procedures 54% do Additional Questions 2015 Decision-making is shared among disciplines in response to changes in patients' conditions or issues that arise during operations (n=753) 19 61 19 Disagreements are resolved with an emphasis not on who is right but what is right for the patient (n=753) 29 53 18 Communication breakdowns frequently lead to delays in starting surgical procedures (n=793) 46 40 15 I am encouraged to report any patient safety concerns I may have (n=836) 18 42 40 0 10 20 30 40 50 60 70 80 90 100 Negative or Neutral Response (1-4) Agree (5-6) Strongly Agree (7)
24 4.3 Differences among Professional Disciplines and Key Demographic Segments Statistical testing was completed on a wide range of segmented groups within the overall data. The mean for each segmented group was compared to the overall mean for each question. This was to understand if there were any significant differences between the groups within a category compared to the overall sample. The statistical testing of the two means was completed by comparing the means at the 95% confidence level, for a 50% response distribution. The following statistically significant differences were identified (results presented below are differences in average scores). Surgeons were more likely to agree that surgical team members are open to changes that improve patient safety, even if it means slowing down (81.5 avg., compared to 69.0) Surgeons were less likely to agree that pressure to move quickly from case to case gets in the way of patient safety (36.6 compared to 52.9) Anaesthetists were less likely to agree that surgical team members all agree on the importance of using checklists (60.9 compared to 74.0) Anaesthetists were less likely to agree that surgical team members from different disciplines always discuss patients conditions and the progress of operations (45.8 compared to 57.4) Anaesthetists were less likely to agree that surgical teams always discuss the operative plan (48.6 compared to 63.0) Surgical team members identifying as Asian were more likely to agree that team discussions (e.g. briefings and debriefings) are common (77.1 compared to 62.3) Surgical team members identifying as Asian were more likely to agree that surgical team members from different disciplines always discuss patients conditions and the progress of operations (72.6 compared to 57.4). There were no other statistically significant differences in results.
25 5: New Zealand compared to the US This section presents the results from the New Zealand survey with the results from the Harvard School of Public Health survey. Scores here represent the percentage of agree and strongly agree scores. Differences of 10% or more in results are noted below. The overall sample size for the US survey was N=1,852. The only limitation to take into consideration when comparing these two sets of results is the different health sector context in each country, specifically in terms of medical liability and how this might impact on the way in which some of the questions are answered.
26 Contextual (Readiness) US surgical team members are more likely to agree or strongly agree: The Time Out is used in every case by every surgical team The Time Out was difficult to implement Interest in checklist implementation is limited to one profession 100.0 94.8 90.0 84.0 80.0 70.0 61.9 70.3 65.0 68.7 74.3 66.6 60.0 50.0 40.0 35.1 30.0 24.3 20.0 10.0 0.0 Surgical team members are open to changes that improve patient safety, even if it means slowing down The 'Time Out' is used in every case by every surgical team The 'Time Out' was difficult to implement Surgical team members all agree on the importance of using checklists in surgery Interest in checklist implementation is limited to one profession e.g. surgery, anaesthesia or nursing NZ USA
27 Interpersonal (Teamwork) Factor 1: Communication There were no differences in the Communication results. 100.0 90.0 80.0 70.0 68.8 69.7 69.4 75.0 60.0 50.0 51.4 54.7 40.0 30.0 20.0 10.0 0.0 Team discussions (e.g., briefings or debriefings) are common. Surgical team members make sure their comments or instructions are heard. Surgical team members share key information as it becomes available NZ USA
28 Factor 2: Coordination US surgical team members were more likely to agree or strongly agree: Surgical team members from different disciplines always discuss patients conditions and the progress of operations 100.0 90.0 80.0 75.4 77.4 74.9 76.0 81.4 78.9 87.8 83.2 70.0 60.0 54.9 50.0 40.0 39.8 30.0 20.0 10.0 0.0 Surgical team members appear eager to help one another Physicians and nurses work together as a wellcoordinated team. Surgeons and anesthesia providers work together as a well-coordinated team. Surgical team members from different disciplines always discuss patients' conditions and the progress of operations Plans for patient care are adapted as needed. NZ USA
29 Factor 3: Respect There were no differences in the Respect results. 100.0 90.0 80.0 76.9 78.5 79.0 80.0 76.9 76.4 70.0 60.0 64.4 63.1 50.0 40.0 30.0 20.0 10.0 0.0 Surgical team members communicate with me in a respectful manner. My input about patient care is well received by other surgical team members I am always treated as a valuable member of the surgical team Potential errors or mistakes are pointed out without raised voices or condescending remarks NZ USA
30 Factor 4: Assertiveness US surgical team members were more likely to agree or strongly agree: That it is difficult to discuss medical mistakes That surgical team members appear to struggle and do not ask one another for help That it is difficult to speak up when they perceive problems with patient care 100.0 90.0 80.0 78.5 77.8 70.0 69.1 60.0 50.0 40.0 32.7 30.0 23.0 20.0 14.5 10.0 0.0 It is difficult to discuss medical mistakes Surgical team mebers appear to struggle and do not ask one another for help. It is difficult to speak up when I perceive problems with patient care NZ USA
31 Factor 5: Clinical Leadership US surgical team members were more likely to agree or strongly agree: That physicians are only open to suggestions from other physicians That physicians maintain a positive tone throughout operations 100.0 90.0 80.0 70.0 60.0 61.7 67.2 64.2 58.7 50.0 40.0 42.1 30.0 20.0 21.1 10.0 0.0 Physicians are only open to suggestions from other physicians Physicians are present and actively participating in patient care prior to skin incision Physicians maintain a positive tone throughout operations NZ USA
32 Practical (Adherence) US surgical team members were more likely to agree or strongly agree: Surgical team members refer to each other by role instead of name NZ surgical team members were more likely to agree or strongly agree: For complex patients or cases, preoperative briefings always include planning for potential problems 100.0 90.0 84.3 80.0 70.0 66.3 60.0 50.0 51.3 56.8 55.3 53.4 44.0 51.9 55.8 40.0 30.0 20.0 10.0 11.9 0.0 Surgical team members refersurgical teams always discuss to each other by role instead of name (e.g., "Nurse" instead of ""Dana"). the operative plan (i.e., more than the location of the incision and name of the procedure) before incision. For complex patients or cases, preoperative briefings always include planning for potential problems Postoperative debriefings always include a discussion of key concerns for patient recovery and post-op management. Equipment issues or other problems discussed in postoperative debriefings are addressed in a timely manner. NZ USA
33 Consequential (Other Items) There were no differences in the Consequential results. 100.0 96.1 92.8 90.0 85.6 85.4 80.0 70.0 60.0 50.0 40.0 38.1 43.8 30.0 20.0 10.0 0.0 I would feel safe being treated here as a patient If I were having an operation, I would want a surgical safety checklist to be used Pressure to move quickly from case to case gets in the way of patient safety NZ USA
34 6: Overview of Qualitative Feedback One open-ended question was included in this survey. Participants were asked if they had any feedback or comments about any of the topics in the survey. Comments were provided by 175 participants. While there were a range of comments made, there were five key themes evident in the qualitative comments: 1. Inattention and/or a lack of engagement and/or a lack of attendance during peri-operative briefing and the checklist 2. Briefing and/or debriefing meetings not universally implemented 3. Communications issues a lack of information sharing and a lack of communications processes/protocols in place 4. Post-operative safety/protocols (including perceptions of surgery time pressures) 5. Team culture issues Examples of comments under each of these key themes are provided below (a full list of qualitative comments was provided in a separate document): Inattention and/or a lack of engagement and/or a lack of attendance during peri-operative briefing/time Out and the checklist When in Theatre, there have been times where the surgical team (surgeons/nurses/techs) are not paying attention during time-out/pre-op briefing and intra-operatively. I find this distracts us from being safe as instructions are not heard and followed, miscommunications happen. Perhaps this topic can be addressed in future surveys (The amount of distraction that occurs pre/intra and post operatively), i.e. non-urgent phone calls (i.e. discharge letters/medical certificates for patients in recovery bay), chats in the background. (Surgeon) Checklists and Time Out procedures seem to be least well used when they are most needed. Anaesthetists and surgeons (speaking as an Anaesthesia trainee) are both guilty of being distracted and distractible during the Time Out process and we easily kid ourselves that this is because we have a very important task at hand when in fact it's often a task that can wait until the end of time out. (Anaesthesia Registrar) Engagement in Time Out and team briefing seems to be inconsistent. Nurses are more likely to engage, doctors not so much. There's an arrogance within some members of the medical team that must be addressed before any of these initiatives can work well. There needs to be consequences for not complying with standard safety precautions and procedures. (PACU Nurse) From personal experience, Surgeons often commence the procedure prior to the Nurse finishing the Time Out segment. Due to the culture of the operating theatre, it is difficult for Nurses to 'speak up' and ask the team to stop what they are doing and be attentive. Another issue has been that several members of the team are engaged in side conversations whilst Time Out is being completed. However, Nurses are becoming more confident in asking for the Team's full attention during this process. (Theatre Nurse)
35 "After 2-3 years since implementation of the surgical check list, nurses at this DHB are still fully responsible for the surgical check list and timeout before incision. We still struggle to get the whole teams attention, struggle to get quietness in the room without resorting to loud calling for participation of the team. The checklist is mainly completed by the nurse often without participation or comment from the team. I find it remains just another tick list that nurses have to fill out. There has been a very slow buy-in from the surgeons and anesthetists at our DBH. (Theatre Nurse) It is changing, but I have been surprised at how long it is taking. It is noted by me, that the younger generation of surgeons are interested and beginning to ask for a time out before incision. They do not appear to be that interested in the other two parts of the checklist, in fact I doubt they know they exist. In our DHB these two parts seem to be just tick boxes for nurses to do." (Theatre Nurse) I think the way things are handled with regards to the check list it really depends on the anaesthetist and surgeon and how receptive they are. Some of them rush and make you feel as though you are holding things up and being difficult when you mention the checklist. (RN) Briefing and/or debriefing meetings not universally implemented Briefings are rarely held. One surgeon always holds them and they are useful. (Anaesthetist) Briefing and debriefing is not currently done at the DHB I currently work for. Whilst our surgeons refer to us by our Christian name, we still feel very valued as team members, however, in the presence of a patient correct and formal address is used. (Theatre Nurse) Currently our operating theatre does not do the briefing and debriefings. We would like to however the clinical lead nominated (a surgeon) has shown no interest in getting it going. There is positive feedback from the anaesthetic team and nursing team to do these debriefings and debriefings but without support from the surgeons it is difficult to achieve. We have an informal session between anaesthetic and nursing each day before the list starts. (Theatre Nurse) Debriefing is rarely done in our theatre. (Theatre Nurse) Different teams behave differently. We can all remember occasions when the team was awful but usually people function well together. Juniors struggle on without asking their seniors for help. We do not have a post op debrief. (Anaesthetist) We are not doing whole team debriefs unless there has been a particularly notable event. (Anaesthetist) Communications issues a lack of communications and a lack of communications processes/protocols in place One final complaint is changes to the operating list that occur but without the surgeon informing the anaesthetist. Surgeons are extremely poor communicators. Wrong patients have been brought into operating rooms because of this. (Anaesthetist) I find that the lack of communication from the surgical team makes it hard to organise gear in a timely manner. We also experience doctors changing their minds on what they want to do during the operation, which then means I have to organise gear straight away, as the patient is already asleep. This put huge pressure on all parties involved (Surgical companies, SSU, theatre staff, myself). (Anaesthetist)
36 Communication re change of surgical plan during operation not always clearly communicated to the WHOLE team (i.e. anaesthetic/circulating nurse or even sometimes scrub nurse). These changes often need to be picked up by team members themselves. (Theatre Nurse) Having to ask multiple times for something leads to a degree of frustration, a raising of voice in case the problem is due to not being heard, and can be a safety issue for patients. (Surgeon) Post-operative safety/protocols (including perceptions of surgery time pressures) As an anaesthetist, the single thing that bugs me most about the WHO SSC is when the theatre nurse asks me Do you have any concerns to hand over for recovery? after the surgeon has finished but while I am still in the process of waking my patient up. This is a critical period and unhelpful interruptions are a distraction. My care of the patient is not yet complete. Furthermore, even if I did have concerns for handover, the theatre nurse would not be in a situation to do anything about it. I would be giving a personal handover to the PACU nurse and would list my concerns for recovery directly to the nurse recovering the patient in PACU. That information is wasted on the theatre nurse. (Anaesthetist) As a nurse in the PACU, I find that handovers from OT are disjointed. Documentation by both the surgical and anaesthetic teams regarding follow-up care is frequently incomplete, requiring repeated phone calls and messages. Patients often arrive poorly positioned in their bed, with lines and tubes jumbled, linen not covering the patient. It seems that the pressure to move the patient quickly from the table and out of theatre means that no one takes responsibility for this. This is particularly problematic in the bariatric patient for whom optimal positioning is so important (PACU Nurse) A constant threat to patient safety in the OT is constant drive by management to improve efficiency in OT. There is an obsession with timings and through-put without equal focus on quality and safety (Anaesthetist) Team culture issues Some surgeons get rude and yell at the rest of the team members, and some have even tossed their equipment on to the trolley and pack a tantrum. Some Doctors are better at receiving and accepting input from someone who is not of their standing than others making it difficult to feel valued as an equal team member (Anaesthetic Technician) There are some surgeons and anaesthetists who are good at working within a team. However, it seems that there is still hierarchical thinking present among some doctors, and they believe that the nurses should just shut up and keep to their little job (actual comment by an anaesthetist after I tried to mention to her that one of our patients had no gag reflex). She also asked me do you have anything important to say?, interrupting me trying to communicate with her about the patient. I find that younger nurses bear the brunt of disrespect and the idea that nurses don't know anything and/or are holding up the list progression. The area in which I work has no briefing before the start of the list, and I think that some surgeons think the nurses don't know what they are doing when they change the surgical plan and haven't communicated it to the nurses (Theatre Nurse)
37 7: Demographic and Other Variables The following section provides the key demographic and other variables of the surgical team members who took part in this survey. 6.1 Person (or one of the people) responsible for surgical safety checklist implementation in the operating theatres where they work Person or one of the people responsible 2015 2015 N=971 Yes 694 71% No 277 29% 6.2 Surgical service(s) worked in Surgical services 2015 2015 N=971 General 501 52% Trauma 334 34% Orthopaedic 409 42% Neurosurgery 126 13% Cardiac 75 8% Thoracic 90 9% Vascular 214 22% Paediatric 251 26% ENT 331 34% Urology 300 31% Gynaecology 403 42% Ambulatory 174 18% Other 319 33% This was a multiple response question
38 6.3 Surgical service(s) worked in most often Surgical services worked in most often 2015 2015 N=971 General 416 43% Trauma 176 18% Orthopaedic 326 34% Neurosurgery 49 5% Cardiac 62 6% Thoracic 49 5% Vascular 85 9% Paediatric 118 12% ENT 183 19% Urology 127 13% Gynaecology 242 25% Ambulatory 84 9% Other 257 26% This was a multiple response question 6.4 Gender Gender 2015 2015 N=756 Male 298 39% Female 428 57% I would rather not say 30 4% 6.5 Age group Age group 2015 2015 N=756 18-25 years of age 33 4% 26-35 175 23% 36-45 210 28% 46-55 179 24% 56+ 122 16% I would rather not say 37 5%
39 6.6 Ethnicity Ethnicity 2015 2015 N=758 Asian 76 10% Asian Indian 37 5% Pacific peoples 10 1% Māori 12 2% New Zealand European 397 52% Other European 100 13% Other 50 7% I would rather not say 76 10% 6.7 Primary professional role Primary professional role 2015 2015 N=761 Anaesthetic Technician 62 8% Anaesthetist 157 21% Intern/Resident/Fellow 62 8% Perfusionist 3 0% Surgeon 128 17% Theatre Nurse 275 36% Other 74 10% 6.8 Number of years worked in this role at any hospital Primary professional role 2015 2015 N=761 Less than 1 year 59 8% Between 1 and 5 years 199 26% Between 6 and 10 years 162 21% More than 10 years 341 45% 6.9 Currently also working for a private hospital Also working for a private hospital 2015 2015 N=756 Yes 160 21% No 596 79%
40 Appendix 1: NZ All Results
41