Hospital Survey on Patient Safety Culture in Slovenia: a psychometric evaluation

International Journal for Quality in Health Care 2013; Volume 25, Number 4: pp. 469 475 Advance Access Publication: 4 June 2013 Hospital Survey on Patient Safety Culture in Slovenia: a psychometric evaluation ANDREJ ROBIDA Centre for Quality and Safety in Healthcare, Bled, Slovenia Address reprint requests to: Andrej Robida. Tel: +386-45-74-01-35; E-mail: andrej.robida@prosunt.si; andrej.robida@telemach.net Accepted for publication 24 April 2013 10.1093/intqhc/mzt040 Abstract Objective. To study the psychometric properties of a translated version of the Agency for Healthcare Research and Quality Hospital Survey on Patient Safety Culture (HSOPSC) in the Slovenian setting. Design. A cross-sectional psychometric study including principal component and confirmatory factor analysis. The percentage of positive responses for the 12 dimensions (42 items) of patient safety culture and differences at unit and hospital-level were calculated. Setting. Three acute general hospitals. Participants. Census of clinical and non-clinical staff (n =976). Main Outcome Measures. Model fit, internal consistency and scale score correlations. Results. Principal component analysis showed a 9-factor model with 39 items would be appropriate for a Slovene sample, but a Satorra-Bentler scaled χ 2 difference test demonstrated that the 12-factor model fitted Slovene data significantly better. Internal consistency was found to be at an acceptable level. Most of the relationships between patient safety culture dimensions were strong to moderate. The relationship between all 12 dimensions and the patient safety grade was negative. The unit-level dimensions of patient safety were perceived better than the dimensions at the hospital-level. Conclusion. The original 12-factor model for the HSOPSC was a good fit for a translated version of the instrument for use in the Slovene setting. Keywords: patient safety culture, survey, psychometric analysis Introduction A safety culture is a basic necessary prerequisite for the improvement of patient safety. To assess the patient safety culture self-reported surveys on patient safety climate are often conducted [1]. They have been used to engage healthcare professionals and leadership in the improvement of patient safety and to establish strategies and programmes of patient safety in their organizations [2]. Patient safety climate is a shared perception among healthcare staff about important characteristics of patient safety. It reflects patient safety culture understood as fundamental values, behaviours and beliefs in a healthcare organization s approach to patient safety [3]. Many countries have begun to measure the perception of patient safety culture of healthcare professionals. In the countries of the European Union, there is also increasing external pressure on healthcare organizations through the Directive on Patients Rights in Cross-Border Healthcare where quality and safety of healthcare services is a priority [4]. Although many instruments for assessing patient safety culture exist [5], a questionnaire developed by Sorra and Nieva [6] has met more psychometric criteria compared with other instruments [7] and has been used in most studies for hospital surveys in the USA, Europe and elsewhere [8 14]. The use of the same instrument would allow for future European comparisons of patient safety culture for patients seeking care in other European countries. This study evaluated the psychometric properties of a Slovenian translation of the Agency for Healthcare Research and Quality supported Hospital Survey on Patient Safety Culture (HSOPSC) questionnaire [7] to assess its appropriateness for Slovenian settings. Methods The instrument The HSOPSC questionnaire consists of 42 items addressing 7 unit-level, 3 hospital-level aspects of safety culture and 4 outcome variables of which 2 (overall patient safety grade and number of events reported in the last 12 months) were International Journal for Quality in Health Care vol. 25 no. 4 The Author 2013. Published by Oxford University Press in association with the International Society for Quality in Health Care; all rights reserved 469

Robida single-item measures. The questionnaire was translated into the Slovene language by a bilingual healthcare professional and by an expert bilingual translator. The draft translation was then pre-tested by several hospital physicians and nurses as well as by non-clinical staff for comprehension. The pre-test findings were appropriately incorporated into the final version of the questionnaire. Adaptations were made only in demographic items concerning departmental structure of the participating hospitals and difference in professional groups. Sample The paper-based questionnaire was distributed to all clinical and non-clinical staff of three acute general hospitals in 2010 (n = 1745) and 1048 questionnaires were returned (response rate of 60%). Of these 72 respondents who answered less than one entire section of the questionnaire, fewer than half of the items throughout the entire survey or where scoring of the items was the same were excluded, leaving 976 questionnaires available for analysis. Statistical analysis The average missing values were 4% (range 0.9% A3 to 8.8% RF5). Pairwise deletion was used to calculate relevant statistics. A Likert scale from 1 (strongly disagree and never) to 5 (strongly agree and always) was used to score the answers. Descriptive statistics were calculated with mean, standard deviation (SD), skewness and kurtosis. The sample was split at random into two independent groups because principal component analysis (PCA) and confirmatory factor analysis (CFA) cannot be performed on the same data set [15]. A PCA was performed on the first half of the sample (n = 501) to study the component structure of the new translation of the original instrument into another language and different cultural setting. A cut-off value for factor loading of 0.4 was chosen to minimize item cross-loadings [16]. CFA was applied next to assess fit of the model to the data in the Slovenian sample [17]. Two separate CFAs were done on the other half of the data set (n = 475) in order to compare the model fit of the original 12-factor versus the alternative model. Two CFA fit indices were employed: the comparative fit index (CFI), with values >0.90 indicating an acceptable fit and >0.95 indicating a good fit and the root mean square error of approximation (RMSEA), with values <0.05/<0.08 considered as an indication of a good/acceptable model fit to the data [18]. To evaluate the difference in fit between the original 12-factor, 42-item model and nested 9-factor, 42-item model the Satorra-Bentler scaled χ 2 difference test was calculated [19]. To assess internal consistency for the 12-factor, 42-item model, the Cronbach α score was calculated using the whole sample, construct validity was assessed by computing scale scores for each factor by calculating the mean score of the items for each particular factor for each respondent. Inter correlations between the scale scores for the 12 factors and the overall patient safety grade were computed to determine the discriminant validity. For the number of events reported, no correlation was calculated due to the skewed nature of this item (43% of the respondents did not report any events and 33% had reported only 1 2 events; n = 976). Strengths were defined artificially as those positively worded items which 75% of respondents endorse by answering agree/strongly agree, or most of the time /always (or when 75% of respondents disagreed with negatively worded items). Areas with the potential for improvement were identified as items which 50% of respondents answered positively. The average positive percentage of each dimension and item with 95% confidence interval was calculated [20]. The z-test was used to calculate the difference between the percentage of positive responses on dimensions of patient safety culture at unit and hospital-level. For statistical analysis SPSS 17, AMOS 18 and EQS 6.1 for Windows statistical packages were used. Results The Kaiser-Meyer-Olkin coefficient was >0.9, indicating common variance among the items and the Bartlett test of sphericity (χ 2 = 7086.2; df = 861; P < 0.001) demonstrating inter-item correlations sufficient for conducting PCA. After exclusion of three items with loadings <0.4 or cross-loading: after we make changes to improve patient safety, we evaluate their effectiveness (A13), staff are afraid to ask questions when something does not seem right (RC6), shift changes are problematic for patients in this hospital (RF11), the resulting PCA model explained 57.5% of the total variance. This model differed from the 12-factor 42-item model in that three factors from the original model (communication openness and feedback and communication about error, staffing and overall perceptions of safety and hospital handoffs and transitions and teamwork across hospital units) collapsed into a single factor. The CFA for the original 12-factor model with 42 items (χ 2 = 3892.3; df 820; P < 0.0005, n = 976) showed CFI was 0.9 and RMSEA was 0.048. The standardized factor loadings were generally large (>0.60) and ranged from 0.21 (organizational learning-continuous improvement) to 0.94 (frequency of event reporting). The alternative 9-factor model with 39 items (χ 2 = 3515.0; df 703; P < 0.0005, n = 976) also fitted adequately with CFI 0.9 and RMSEA 0.049. The standardized factor loadings were also generally large (>0.60) and ranged from 0.2 (organizational learning-continuous improvement) to 0.93 (frequency of event reporting). Next, CFA was computed for the nested 9-factor model with adding of the three items (A13, RC6 and RF11) that were dropped from the 9-factor, 39-item model. This nested model (χ 2 = 3119.2; df 595; P < 0.0005, n = 976) likewise had an acceptable fit (CFI was 0.9 and RMSE was 0.050). The standard factor loading ranged from 0.21 (organizational learning continuous improvement) to 0.92 (frequency of event reporting) with generally high loadings (>0.60). To evaluate the difference in fit between the original 12-factor, 42-item model and nested 9-factor, 42-item model the Satorra-Bentler scaled χ 2 difference test was calculated [19]. The Satorra-Bentler scaled χ 2 difference test showed that the 12-factor model with 42 items was a significantly better fit than the more constrained 470

Patient safety culture Safety and management 9-factor nested model with 42 items (Satorra-Bentler scaled χ 2 difference = 117.428; df 30; P < 0.001). The Cronbach α score of the 12 scales (Table 1) indicated an acceptable level of internal consistency (>0.70) except for hospital handoffs and transitions, staffing, non-punitive response to error (0.61 0.66). Cronbach s alpha score was low for organizational learning and continuous improvement (0.36). George et al. [21] provide the following rules of thumb regarding levels of internal consistency: >0.9, excellent; >0.8, good; >0.7, acceptable; >0.6, questionable; >0.5, poor and <0.5, unacceptable. A comparison of internal consistency with some other countries is also given in Table 1. Intercorrelation coefficients between HSOPSC scales were significant (P < 0.01). The highest correlation was between feedback and communication about error and communication openness (0.63). Overall patient safety grade weakly to strongly and negatively correlated with all 12 factors (Table 2). The highest positive percentage score was on frequency of events reported and the lowest on staffing (Table 3). The unit-level dimensions (54% of positive responses) of patient safety are perceived better than the dimensions at hospital-level (44% of positive responses), z-score = 4.995 (P < 0.01). More than 80% of respondents graded patient safety as good to acceptable. Forty-three percent did not report any event in the 12-month period (Table 4). Discussion Before the adoption of HSOPSC questionnaires from different cultural settings and languages a psychometric analysis is necessary to see if the questionnaire needs modifications. Psychometric properties of the original AHRQ questionnaire have been validated in US hospital settings and exploratory factor analysis resulted in 12 dimensions [6]. In different European countries exploratory factor analysis and CFA showed that some modifications were necessary. The Dutch translated version of HSOPSC showed an 11-factor model [11], the German version disclosed an 8-factor model [12], the UK sample showed a 9-factor model [13] and the Scottish sample revealed a 10-factor model [14]. In the present study three models were explored: the original AHRQ 12-factor, 42-item model; the 9-factor, 39-item model and nested 9-factor, 42-item model to see how they fit the Slovenian data. PCA findings indicated an alternative 9-factor model which only slightly differ from the original 12-factor model. Furthermore, the Satorra-Bentler scaled χ 2 difference test demonstrated that a 12-factor model significantly better fit our data. This was similar to Sarac et al. [14] where the difference between their 10-factor model and the original 12-factor model also showed the latter better fit their data. In other studies, Satorra-Bentler scaled χ 2 difference test was not computed [11 13]. In view of possible cross-border comparison of patient safety culture no modification to the original 12-factor model is necessary and thus it would better serve to judge patient safety in Slovene hospitals when mobility of patients in the European Union becomes a reality in 2014. The Cronbach α score of the 12 scales indicated an acceptable level of internal consistency for most of the dimensions. Findings were comparable with other data [9 14], except for organizational learning and continuous improvement where α score was low. Internal consistency: scores of 0.6 are considered acceptable, and <0.5 unacceptable [21]. CFA revealed a low standardized regression weight for the item: mistakes have led to Table 1 Internal consistency Scales (number of items) Cronbach α SI SCT US NL NO UK BE CH...... Unit-level safety culture dimensions Supervisor/manager expectations (4) 0.74 0.79 0.75 0.70 0.77 0.68 0.77 0.79 Organizational learning and continuous improvement (3) 0.36 0.64 0.76 0.57 0.51 0.66 0.59 0.68 Teamwork within units (4) 0.74 0.80 0.83 0.66 0.77 0.73 0.66 0.75 Communication openness (3) 0.74 0.73 0.72 0.72 0.68 0.67 0.65 0.64 Feedback and communication about error (3) 0.72 0.78 0.78 0.75 0.70 0.80 0.78 0.78 Non-punitive response to error (3) 0.61 0.77 0.79 0.69 0.64 0.65 0.68 0.70 Staffing (4) 0.65 0.60 0.63 0.49 0.65 0.58 0.57 0.65 Hospital-level safety culture dimensions Hospital management support for patient safety(3) 0.82 0.79 0.83 0.68 0.79 0.69 0.72 0.83 Teamwork across hospital units (4) 0.74 0.70 0.80 0.59 0.65 0.70 0.66 0.77 Hospital handoffs and transitions (4) 0.66 0.74 0.80 0.69 0.65 0.77 0.71 0.72 Outcome measures Overall perceptions of safety (4) 0.65 0.84 0.84 0.79 0.82 0.83 0.85 0.76 Frequency of event reporting (3) 0.88 0.71 0.74 0.62 0.76 0.67 0.58 0.70 SI, Slovenia; SCT, Scotland [14]; US, United States of America [8]; NL, Netherlands [11]; NO, Norway [10]; UK, United Kingdom [13]; BE, Belgium [9]; CH, Switzerland [12]. 471

Robida Table 2 Mean, SD and intercorrelation coefficients for 12-factor patient safety culture and patient safety grade Factor Composite of patient safety culture Mean SD FA1 FA2 FA3 FA4 FA5 FA6 FA7 FA8 FA9 FA10 FA11 FA12... FA1 Overall perceptions of safety 3.45 0.73 FA2 Frequency of events reported 3.94 0.98 0.22 FA3 Supervisor/manager expectations and actions promoting patient safety 3.64 0.69 0.51 0.18 FA4 Organizational learning continuous improvement 3.59 0.55 0.40 0.20 0.38 FA5 Teamwork within units 3.50 0.72 0.37 0.16 0.43 0.42 FA6 Communication openness 3.72 0.82 0.41 0.27 0.54 0.38 0.44 FA7 Feedback and communication about error 3.59 0.79 0.43 0.32 0.49 0.47 0.45 0.63 FA8 Non-punitive response to error 3.13 0.76 0.38 0.12 0.38 0.25 0.27 0.38 0.29 FA9 Staffing 2.80 0.78 0.50 0.08 0.36 0.22 0.23 0.23 0.22 0.39 FA10 Hospital management support for patient safety 3.11 0.87 0.43 0.19 0.37 0.37 0.40 0.32 0.39 0.34 0.34 FA11 Teamwork across hospital units 3.11 0.87 0.43 0.19 0.36 0.32 0.40 0.36 0.40 0.21 0.26 0.53 FA12 Hospital handoffs and transitions 3.40 0.66 0.38 0.18 0.26 0.18 0.24 0.29 0.25 0.22 0.29 0.32 0.55 Safety grade 0.52 0.16 0.30 0.31 0.26 0.29 0.30 0.20 0.34 0.37 0.36 0.30 P < 0.01; FA, factors. positive changes here. This was similar to the findings of Olsen [10]. Removing of this item may be appropriate but could diminish the opportunity for benchmarking. The relationship among patient safety culture dimensions demonstrated strong correlation between supervisor/manager expectations/actions promoting patient safety and overall perceptions of safety, between supervisor/manager expectations/actions promoting patient safety and communication openness, between hospital management support for patient safety and teamwork across hospital units and between hospital handoffs/transitions and teamwork across hospital units. Other relationships were weak to moderate [22]. The relationship between all 12 dimensions and the patient safety grade was negative indicating that this outcome variable is in contradiction with staff perception on the 12 dimensions of patient safety culture and might have reflected the staff perception of patient safety grades more positively than the rest of patient safety culture dimensions. None of the patient safety culture dimensions reached the artificially set value of 75% of positive answers. There were also some contradictory results, for example, between frequency of events reported (69% of positive answers) and non-punitive response to error (39% of positive answers). This difference may be explained by the understanding of the importance by the hospital staff to report errors on the one hand, and fear of the prevailing blame and shame culture in Slovenia [23] on the other hand, as unintentionally made errors may be and indeed are prosecuted in criminal courts and licenses of healthcare professionals might be temporarily withdrawn. Several areas of HSOPSC suggested opportunities for improvement in safety culture: non-punitive response to error, staffing, hospital management support for patient safety and teamwork across hospital units. These dimensions received 50% of positive responses. The score on staffing was particularly low. This reflects the perception of respondents that problems for patient safety are mainly due to inadequate staffing. Although staffing is important [24], it is not the sole solution to problems of patient safety. The study showed that survey s items and dimensions are psychometrically sound at the individual level. Sorra and Dyer [8] found that the same is true at the unit and hospital levels. The perception of patient safety culture dimensions at unit-level was significantly better than at hospital-level. This could reflect either better cooperation and teamwork at unit-level or socially desirable answers where respondents tend to give positive self-descriptions of their unit [25, 26]. Further studies are necessary to investigate this in other cultural settings. The study was limited to self-reported outcome variables and was not validated against objectively measured frequency of adverse events and near misses. The present study is a foundation for raising awareness of problems of patient safety among staff. Although the study was limited to self-reported outcome variables and was not validated against objectively measured frequency of adverse events and near misses, the results can help hospitals introduce a system approach to patient safety, plan patient safety solutions, follow changes in patient safety culture over time and eliminate dangerous practices of name, blame and shame culture. To improve patient safety a reporting and learning 472

Patient safety culture Safety and management Table 3 Scores for 12 dimensions patient safety culture for a Slovene sample (n = 976) Dimensions and items of patient safety culture Score 95% confidence interval... Overall perceptions of safety 56 53 59 Patient safety is never sacrificed to get more work done (A15) 65 61 66 Our procedures and systems are good at preventing errors from happening (A18) 50 46 53 It is just by chance that more serious mistakes do not happen around here (RA10) 57 53 60 We have patient safety problems in this unit (RA17) 55 51 58 Frequency of events reported 69 66 72 When a mistake is made, but is caught and corrected before affecting the patient, how often is this 72 69 75 reported? (D1) When a mistake is made, but has no potential to harm the patient, how often is this reported? (D2) 61 58 64 When a mistake is made that could harm the patient, but does not, how often is this reported? (D3) 74 71 77 Supervisor/manager expectations and actions promoting patient safety 65 62 8 My supervisor/manager says a good word when he/she sees a job done according to established 51 47 54 patient safety procedures (B1) My supervisor/manager seriously considers staff suggestions for improving patient safety (B2) 68 65 71 Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means 63 60 66 taking shortcuts (RB3) My supervisor/manager overlooks patient safety problems that happen over and over (RB4) 80 77 82 Organizational learning continuous improvement 64 61 67 We are actively doing things to improve patient safety (A6) 80 78 83 Mistakes have led to positive changes here (A9) 58 55 61 After we make changes to improve patient safety, we evaluate their effectiveness (A13) 53 50 56 Teamwork within units 61 58 64 People support one another in this unit (A1) 64 61 67 When a lot of work needs to be done quickly, we work together as a team to get the work done (A3) 82 80 84 In this unit, people treat each other with respect (A4) 54 51 57 When one area in this unit gets really busy, others help out (A11) 44 41 47 Communication openness 61 57 64 Staff will freely speak up if they see something that may negatively affect patient care (C2) 68 65 79 Staff feel free to question the decisions or actions of those with more authority (C4) 60 57 63 Staff are afraid to ask questions when something does not seem right (RC6) 53 50 57 Feedback and communication about error 56 53 57 We are given feedback about changes put into place based on event reports (C1) 30 27 33 We are informed about errors that happen in this unit (C3) 64 61 67 In this unit, we discuss ways to prevent errors from happening again (C5) 74 71 76 Non-punitive response to error 39 36 42 Staff feel like their mistakes are held against them (RA8) 41 38 44 When an event is reported, it feels like the person is being written up, not the problem (RA12) 42 39 45 Staff worry that mistakes they make are kept in their personnel file (RA16) 35 32 38 Staffing 31 29 34 We have enough staff to handle the workload (A2) 23 21 26 Staff in this unit work longer hours than is best for patient care (RA5) 27 24 30 We use more agency/temporary staff than is best for patient care (RA7) 42 40 46 We work in crisis mode trying to do too much, too quickly (RA14) 32 29 35 Hospital management support for patient safety 39 36 42 Hospital management provides a work climate that promotes patient safety (F1) 41 38 44 The actions of hospital management show that patient safety is a top priority (F8) 39 36 42 Hospital management seems interested in patient safety only after an adverse event happens (RF9) 37 34 40 Teamwork across hospital units 41 40 46 There is good cooperation among hospital units that need to work together (F4) 48 45 51 Hospital units work well together to provide the best care for patients (F10) 44 41 47 Hospital units do not coordinate well with each other (RF2) 20 18 23 (continued ) 473

Robida Table 3 Continued Dimensions and items of patient safety culture Score 95% confidence interval... It is often unpleasant to work with staff from other hospital units (RF6) 53 49 56 Hospital handoffs and transitions 53 50 57 Things fall between the cracks when transferring patients from one unit to another (RF3) 58 55 61 Important patient care information is often lost during shift changes (RF5) 71 68 74 Problems often occur in the exchange of information across hospital units (RF7) 38 35 41 Shift changes are problematic for patients in this hospital (RF11) 46 43 50 R, reversed items. Table 4 Overall patient safety grade and number of events reported and submitted in the last 12 months... atmosphere should be created. Support from the government must be obtained to achieve non-punitive approach to patient safety with protection of those who report errors [27]. Funding This study was supported by the Ministry of Health of the Republic of Slovenia [N2711-10-0699]. References % of respondents (n = 976) Overall patient safety grade a Excellent 9 Good 47 Acceptable 38 Poor 5 Failure 1 Number of events reported and submitted in the last 12 months b None 43 1 2 33 3 5 14 6 10 7 11 20 1 21 1 a 9% and b 13% of respondents did not answer. 1. Flin R. Measuring safety culture in healthcare: a case for accurate diagnosis. Saf Sci 2007;45:653 667. 2. Pronovost P, Weast B, Holzmueller CG et al. Evaluation of the culture of safety: a survey of clinicians and managers in an academic medical center. Qual Saf Healtcare 2003;12:405 410. 3. Denison DR. What is the difference between organizational culture and organizational climate? A native point view on a decade of paradigm wars. Acad Manag Rev 1996;21:619 654. 4. Directive 2011/24/EU of the European parliament and of the council on the application of patients rights in cross-border healthcare. Official Journal of the European Union L 88/45, Luxembourg: Publication Office of the European Union, 2011, p. 3. 5. Singla AK, Kitch BT, Weissman JS et al. Assessing patient safety culture: a review and synthesis of the measurement tools. J Patient Saf 2006;2:105 115. 6. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: AHRQ Publication No. 04-004, 2004. 7. Flin R, Burns C, Mearns K et al. Measuring safety climate in healthcare. Qual Saf Health Care 2006;15:109 115. 8. Sorra JS, Dyer N. Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture. BMC Health Serv Res 2010;10:199. 9. Hellings J, Schrooten W, Klazinga N et al. Challenging patient safety culture: survey results. Int J Health Care Qual Assur 2007;20:620 632. 10. Olsen E. Reliability and validity of the hospital survey on patient safety culture at a Norwegian hospital. In: Øvretveit J, Sousa P, (eds). Quality and Safety Improvement Research. Lisbon: National School of Public Health, 2008, pp. 173 86. 11. Smits M, Dingelhoff IC, Wagner C et al. The psychometric properties of the Hospital Survey on Patient Safety Culture in Dutch hospitals. BMC Health Serv Res 2008;8:230. 12. Pfeiffer Y, Manser T. Development of the German version of the hospital survey on patient safety culture: dimensionality and psychometric properties. Saf Sci 2010;48:1452 62. 13. Waterson PE, Griffiths P, Stride C et al. Psychometric properties of the hospital survey on patient safety: findings from the UK. Qual Saf Health Care 2010;19:1 5. 14. Sarac C, Flin R, Kathryn Mearns K et al. Hospital survey on patient safety culture: psychometric analysis on a Scottish sample. BMJ Qual Saf 2011;20:842 848. 15. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Testing Structural Equation Models. Thousand Oaks: SAGE, 1993. 474

Patient safety culture Safety and management 16. Costello AB, Osborne JW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval 2005;10:1 9. 17. Worthington RL, Whittaker TA. Scale development research: a content analysis and recommendations for best practices. Couns Psychol 2006;34:806 838. 18. Hu L-T, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Model 1999;6:1 55. 19. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika 2001;66: 507 514. 20. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 1988;17:857 872. 21. George D, Mallery P. SPSS for Windows Step by Step: A Simple Guide and Reference. Boston: Allyn & Bacon, 2003. 22. Cohen JW. Statistical Power Analysis for the Behavioral Sciences. Hilldadale: Lawrence Erlbaum Ass., 1988. 23. Robida A. Sentinel events (in Slovene). Zdrav Vestn 2004;73:681 687. 24. Aiken LH, Clarke SP, Sloane DM et al. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002;288:1987 93. 25. Delroy L, Paulhus DL. Socially desirable responding: the evolution of a construct. In: Brown HI, Jackson DN, Wiley DE (eds). The Role of Constructs in Psychological and Educational Measurement. Mahwah: Lawrence Erlbaum, 2002. 26. Tourangeau R, Yan T. Sensitive questions in surveys. Psychol Bull 2007;133:859 83. 27. The Ministry of Interior and Health, Denmark. Danish Health Care Act no. 913 of 13/07/2010, Denmark. http:// patientsikkerhed.dk/index.php?id=471 (8 May, 2013, date last accessed). 475