Measuring Nursing Quality Prof. Dr. Michael Simon Institute of Nursing Science www.nursing.unibas.ch Directorate of Nursing/AHP Nursing Research Unit
Content Background Quality Measurement Example USA: NDNQI Example UK: Safety Thermometer Example GER: PTVS Outlook and conclusions Prof. Dr. Michael Simon 2
What s constitutes a great indicator? I 1. Importance of the Measure Relevance for stakeholders Health importance Potential for improvement Susceptibility of being influenced Prof. Dr. Michael Simon 3
What s constitutes a great indicator? II 2. Scientific Soundness Explicitness of evidence Strength of evidence Reliability Validity Allowance for stratification or case-mix adjustment if appropriate. Comprehensible Prof. Dr. Michael Simon 4
What s constitutes a great indicator? III 3. Feasibility Explicit specification Data availability Prof. Dr. Michael Simon 5
Florence Nightingale Nightingale, Florence. (1863). Notes on hospitals (3d ed.). London: Longman, Roberts, and Green. Prof. Dr. Michael Simon 6
Nursing Sensitivity Nursing-sensitive performance measures are processes and outcomes and structural proxies for these processes and outcomes (e.g., skill mix, nurse staffing hours) that are affected, provided, and/or influenced by nursing personnel, but for which nursing is not exclusively responsible. Nursingsensitive measures must be quantifiably influenced by nursing personnel, but the relationship is not necessarily causal. National Quality Forum. National Voluntary Consensus Standards for Nursing-Sensitive Care: An Initial Performance Measure Set. Washington, DC: NQF; 2004. 7 Prof. Dr. Michael Simon 7
Donabedian s S-P-O model Donabedian, A. (1992). "The role of outcomes in quality assessment and assurance." QRB Qual Rev Bull 18(11): 356-360. Prof. Dr. Michael Simon 8
Public Reporting Model BERWICK, D. M., JAMES, B. & COYE, M. J. 2003. Connections between quality measurement and improvement. Medical care, 41, I30-8. Prof. Dr. Michael Simon 9
The bitterness of poor quality remains long after the sweetness of low price is forgotten. Benjamin Franklin Prof. Dr. Michael Simon 10
No of falls per hospital fall Min. : 17.00 1st Qu.: 33.00 Median : 55.00 Mean : 57.62 3rd 11 Qu.: 80.00 Max. :112.00 Prof. Dr. Michael Simon 11
No of falls per hospital Patient days Min. :4481 1st Qu.:4967 Median :5362 Mean :5296 3rd 12 Qu.:5644 Max. :6074 Prof. Dr. Michael Simon 12
Falls per 1,000 patient days Hosp. A Hosp. B fall.rate Min. : 3.034 1st Qu.: 6.503 Median :10.186 Mean :10.944 3rd Qu.:14.604 Max. 13 :22.242 Prof. Dr. Michael Simon 13
Hosp. A Quality Measurement for Selection (Provider Profiling) Concepts Hosp. B x Observed Differences Mean on organisa- Ponal level StraPficaPon PaPent characterispcs Unexplained differences Risk standardizapon, O/E OLS, GLM Empirical Bayes MulPlevel, GLMM Chance/ Uncertainty Unexplained differences Modifiziert nach VAN DISHOECK, A.-M., LINGSMA, H. F., MACKENBACH, J. P. & STEYERBERG, E. W. 2011. Random variapon and rankability of hospitals using outcome indicators. BMJ Qual Saf, 20, 869-874. Residuales Confounding RegistraPon Bias Quality of Care Methods Prof. Dr. Michael Simon 14
Quality Measurement for Quality Improvement t 2 Concepts Chance Hosp A t 1 Observed differences PaPent CharacterisPcs Residuales Confounding Quality of Care t 3 Pme StaPsPcal Process Control (SPC) Methods Regression Prof. Dr. Michael Simon 15
IN GOD WE TRUST; ALL OTHERS MUST BRING DATA. W. EDWARDS DEMING 16 Prof. Dr. Michael Simon 16
Example: % of patients with fall event 1. Crude mean 2. Risk-adjusted standardized rate 3. Empirical Bayes Prof. Dr. Michael Simon 17
Simulated data 50 units 1500-2000 patients per hospital Fall risk: 0-73% Mean fall probability: 0.03 Length of stay: Poisson, mean=3 Prof. Dr. Michael Simon 18
Plain mean: % patient with a fall Min. :0.009 1st Qu. :0.019 Median :0.030 Mean :0.033 3rd Qu. :0.043 Max. :0.065 Prof. Dr. Michael Simon 19
+ Standardized Rate (risk adjusted) Min. :0.009 1st Qu. :0.019 Median :0.030 Mean :0.033 3rd Qu. :0.043 Max. :0.065 Min. :0.020 1st Qu.:0.026 Median :0.028 Mean :0.033 3rd Qu.:0.041 Max. :0.059 Prof. Dr. Michael Simon 20
+ Empirical Bayes (sample size + risk adjusted) Min. :0.009 1st Qu. :0.019 Median :0.030 Mean :0.033 3rd Qu. :0.043 Max. :0.065 Min. :0.020 1st Qu.:0.026 Median :0.028 Mean :0.033 3rd Qu.:0.041 Max. :0.059 Min. :0.018 1st Qu.:0.023 Median :0.025 Mean :0.024 3rd 21 Qu.:0.026 Max. :0.030 Prof. Dr. Michael Simon 21
Summary Under given assumptions: Plain fall rates over estimate differences Risk-adjusted rates have smaller differences Empirical Bayes when fall rates are intended to be used for provider profiling Prof. Dr. Michael Simon 22
Example USA NDNQI Prof. Dr. Michael Simon 23
National Database of Nursing Quality Indicators (NDNQI ) Founded 1998 by the American Nurses Association (ANA) Administered by University of Kansas School of Nursing 05/2014 sold to Press-Ganey ~2000 hospitals >20.000 units Quarterly benchmark reports Annual nurse survey (n=>400,000) NQF-developer Relevant for CMS payment Prof. Dr. Michael Simon 24
Skill mix: Registered Nurses (RNs) Licensed Practical/Vocational Nurses (LPN/LVNs) Unlicensed Assistive Personnel (UAP) Nursing Hours per Patient Day RN Education/Certification Nurse Turnover RN Survey: Practice Environment Scale Job Satisfaction Scales NDNQI-Indicators I Prof. Dr. Michael Simon 25
Patient Falls/Injury Falls Hospital/Unit-Acquired Pressure Ulcers Physical/Sexual Assault Pain Assessment/Intervention/Reassessment Cycles Peripheral IV Infiltration NDNQI-Indicators II Physical Restraints Healthcare-Associated Infections: Catheter-Associated Urinary Tract Infection Central Line-Associated Bloodstream Infection Ventilator-Associated Pneumonia Ventilator-Associated Events Prof. Dr. Michael Simon 26
Examples UK SAFETY THERMOMETER Prof. Dr. Michael Simon 27
Example UK: Safety Thermometer Harm free care Since 2012 P4P (CQUIN) Pressure ulcers, falls, CAUTI, VTE <10 Minutes per patient 1.8 Mio patients in 2013 Open data Prof. Dr. Michael Simon 28
Table 1: NHS Safety Thermometer Patient data Field Values Notes Age Band: 1 = <18, 2 = 18-70, 3 = >70 Sex: 1 = F, 2 = M Old PUs 1 = None, 2 = Cat 2, 3 = Cat 3, 4 = Cat 4 New PUs 1 = None, 2 = Cat 2, 3 = Cat 3, 4 = Cat 4 Falls UTIs Catheters VTE Risk Assessment VTE Prophylaxis VTE Treated 1 = No Fall, 2 = No Harm, 3 = Low Harm, 4 = Moderate Harm, 5 = Severe Harm, 6 = Death 1 = No UTI, 2 = Old UTI, 3 = New UTI 1 = No Catheter, 2 = 1-28 days, 3 = >28 days, 4 = Days Not Known 1 = No, 2 = Yes, 3 = N/A 1 = No, 2 = Yes, 3 = N/A 1 = No VTE, 2 = Old DVT, 3 = Old PE, 4 = Old Other, 5 = New DVT, 6 = New PE, 7 = New Other A value of 2 is not a Harm Values 5 to 7 are New Harms Values 2 to 4 are not Harms HSCIC (2014). NHS Safety Thermometer: Patient Harms and harm free care Prof. Dr. Michael Simon 29
Falls in general medical wards ~2/3 of wards don t have any falls ~3% of patients fell ~1% of patients falls with injuries Prof. Dr. Michael Simon 30
Hospital level comparison Org.Code RHM RXP RXF RWG RFR RTX RAJ RWD RQM RP5 RJN RNQ RNA RM2 R1G RJ2 RYR RBN RGT RXR RXC RH5 RK5 RVL RLT RQ6 VLYJ5 RWJ RBA RNZ RTE RTK RA4 RA3 RV3 REF RVJ RJZ RXW RFW RGN RVV RD8 RJ6 RDU RKE RHW RWN RM1 RA2 RWF RVR RBK RFF RQW RKB RRF RC1 RXK R1H RR7 RXQ RBD RN7 RNL RAS RJD RWP RN3 VM0C0 RYW RJE RJ1 RWW RY5 REM RDD RCX RWY RDZ RCF RC3 RA9 RM3 RA7 RTG RWH RH8 RD1 RC9 RDE RBL RLQ RTD RMP RL4 RGR RW3 RVY RFS RLN RBT RJF RTP (Intercept) 0.5 0.0 0.5 1.0 Prof. Dr. Michael Simon 31
Ward level comparison ward.id RXF:STROKE RXF:MONUMENT RWG:GREEN RXC:CUCKMERE RVR:STH RTX:RLI RWD:GRANTHAM R1G:NEWTON RVV:KCH RWD:PILGRIM RC1:ELIZABETH ACUTE RKE:MERCERS RQM:NELL RTX:FGH RNZ:DURRINGTON RH5:WELLINGTON RH5:CATHEDRAL RVL:CDUMAU REHAB RAJ:ROCHFORD RJ2:CHESNUT RA3:CHEDDAR RAJ:ESTUARY RWG:PURPLE RJZ:BROM RLT:MARY RBT:WARD RYR:EASTBROOK REF:MED RXP:WEARDALE RH5:MARSHFIELD RA7:BRI RLT:ELIZABETH RRF:STANDISH HARBLEDOWN MEDICAL RJ6:PURLEY RP5:WARD RTE:GUITING RJN:WARD RXP:51 RNQ:MAU RFW:LAMPTON RNL:LARCH RWF:MAU RJE:WARD RWP:WAVON2 RBK:WARD ABBOT R1G:TOTNES RXP:4 UNIT RWJ:A10 LEVEL RXP:5 RJ2:ASPEN RV3:WICU GWYNNE RHM:D5 RHM:D6 RHM:D7 RHM:D8 RXR:B4 RFR:A2 HOUSE RXF:A1 RQW:WINTER RP5:WARD VLYJ5:GENERAL RM3:SHC RWD:LINCOLN RHW:REDLANDS RNL:PATTERDALE RH5:MARY RWH:STANBOROUGH RQW:HENRY RWD:LINCOLN RAS:GRANGE ADMISSIONS WARD RW3:WARD RWG:CROXLEY ESCALATION RJ6:HEATHFIELD RYR:BROOKLANDS RXC:FOLKINGTON RXP:SEDGEFIELD RJZ:ANNIE RNL:WILLOW RXW:AMU RMP:WARD RR7:MAU RA4:WARD8A RD8:WARD RRF:LOWTON RVJ:WARD RWF:MERCER R1G:DAWLISH RP5:C2 CARLTON RWD:GRANTHAM RN3:TEAL RYR:AMU RWD:PILGRIM RFW:OSTERLEY RTD:RVI RLQ:LUGG RDD:M RTE:HAZELTON RYR:EARTHAM CARDIO RQM:DAVID RYR:BOXGROVE RTD:RVI RAJ:ELEANOR RTD:FH RYR:PETWORTH RJ2:MULBERRY R1G:PAIGNTON RLT:BOB RJZ:MARY RCX:STANHOE RVJ:WARD RCF:WARD REF:CARNKIE RXP:11 RVJ:WARD ROBERTSON RBD:HINTON RBA:WARD RC9:WARD RJN:WARD RNA:C7 RNA:C8 RWP:WLAU3 RXF:10 RCX:TILNEY RTK:MAPLE MEDICAL RDD:FNMB RTE:GW RQM:AAU RGN:A8 RXR:D1 RBN:2D RM2:F4 RK5:36 DMH RGT:K3 L2 RTE:ACUA &GREEN RGT:D10 RM2:F11 RHM:E2 RQ6:7A 1 TEIGN RDU:AMU R1H:MARY NAVENBY RWW:C22 RLT:CCU GASTRO RDU:F15 RGT:R2 RFR:A4 RFR:A1 GARTH 3 WARD RKB:1 UHND PGI 27 1 UHND WCH UNIT BH1 9 2 AMU TWH DDH 230 8A 15 171 6 RD8:WARD TRAFFORD R1H:11E RWH:9A MOORE RNA:C6 RXR:C7 RVR:C5 RFF:17 TRAUMA RBK:WARD RP5:MAU RC3:5 RYR:LAVANT RJ2:ALDER RTK:MSSU WARREN RGT:C5 RM2:A7 RBN:1B PILLAR RR7:22 RR7:12 ZUNZ UNIT ST PRH SSU AB 21 46 RA2:TILFORD RXQ:MEDICAL RW3:OMU OBSERVATIONAL RWD:LINCOLN RWG:SARRATT RDE:LAYER RWN:HMP RKE:MARY RW3:WARD RN3:SHALBOURNE RNQ:HARROWDEN RWY:WARD RJ2:QEW RNQ:HC RAJ:STAMBRIDGE RAJ:BLENHEIM SHORT RCF:WINTER RVV:WHH RTD:RVI RTX:RLI RW3:WARD RVJ:WARD RXF:MAU RP5:WARD RA4:WARD RA3:BERROW RXP:6 RXP:3 RTX:FGH RM3:CSS R1G:TEIGNMOUTH RC1:ARNOLD MEDICAL WHITCHURCH RC1:GODBER RWF:CORNWALLIS RM3:NR RAS:BEVAN ADMISSIONS RYR:BURLINGTON RTP:CHARLWOOD RM3:SURG SEACOLE RV3:WOODHILL RH5:BRENDON RJ6:FAIRFIELD RJZ:LONSDALE RVJ:K RMP:WARD RH5:ATHLONE 16 RYR:BOSHAM RYW:MHH RYW:WARD RXP:41 RXP:43 RNZ:PITTON RN7:EBONY RVL:LARCH RN7:BEECH RM1:GUIST RWP:AECU RICHARDS RXW:22 RXP:2 RXF:6 RXF:5 RXP:1 STAY PRETTY BEDFORD RQ6:AMU WARD ERSKINE RWP:A12 RGN:B12 R1H:11C COLEBY & RXR:C2 RTE:9B RFS:13 RFS:17 206 A2 2 43 WARD UHND 30 1 WARD MARNEY RP5:ATC RWW:A3 NORTH RXK:L5 RWP:A6 HOBBS RFS:18 RK5:34 ENDO JAKIN RAY DRI 106 UNIT 7B 14 48 07 2 RXF:20 RC3:8 RJZ:TWINING RNA:C8 TRAFFORD MED. RJD:AMU RWH:10BS RRF:CCU RTK:MAU RLT:ADAM RM3:SHC RGR:G9 RJZ:BROM RP5:WARD RM3:SHC RTP:CAPEL RLQ:REDBROOK RGN:B1 RA7:BRI RA7:BRI RJ6:FAIRFIELD RTE:KEMERTON RAS:DRAYTON RC9:WARD R1G:BRIXHAM RW3:WARD RH8:DYBALL RVJ:F RWP:WLAU2 H3 RXW:32E OXFORD RTG:PIU RWJ:C2 RBN:5D 6A RW3:AM3 ANNEXE RWJ:B2 RTE:9A RL4:A7 WARD UHND RL4:C17 NORTH WARD SUITE MEAU RFS:3 DMH SBH SMH CRH DDH EAU CLS SR UNIT PGH 31M 31R 205 C1 9B 15 30 51 16 39 CA RWG:AAU RRF:WINSTANLEY RW3:WARD RXP:RICHARDSON RA3:HARPTREE RTE:WOODMANCOTE RWG:AAU RC1:HARPUR RYR:ERRINGHAM RWD:PILGRIM RC1:WHITBREAD RXQ:WARD RNZ:WHITEPARISH R1H:STRATFORD RAS:PINEWOOD (FORMERLY BEDE RWF:ROMNEY RH8:OKEMENT L6 RH5:EXMOOR RNL:BEECH RJF:WARD RV3:TOPAS RWW:B18 RBL:AMU RTK:MAY RR7:14A RTG:409 SOUTH RFF:23 RH5:COATES MEDICAL R1H:NIGHTINGALE RKE:EXTRA RH5:HADSPEN RJF:WARD RJ1:ALBERT ISOLATION RD1:MSSU RWH:SSU RD8:CDU H2 RBL:AAU RENAL H8 DERM ST W47 RYR:DITCHLING RMP:WARD RHW:SIDMOUTH RXC:SEAFORD RJN:MEDICAL RVR:STH RXP:RICHARDSON RW3:ACUTE RTP:TANDRIDGE RYR:AMU RGN:CARDIAC MARY RVV:KCH RJ6:PURLEY RW3:WARD RQW:HARVEY RWP:WAVON4 RRF:ASTLEY RNA:C8 RNA:C7 RWP:AMAUF RM3:SHC L3 DIAB ASSESSMENT RY5:MANBY RN7:ROSEWOOD RH5:SWINBANKS RDE:LANGHAM MOORE MEDICAL RH8:LOWMAN RC9:WARD RBT:WARD RKB:HOSKYN RJZ:OLIVER RWG:GADE R1H:CURIE 1 RH5:LUKE RDZ:RB24 RW3:AM4 RKB:CDU RRF:ASU RXR:C11 RWH:6B (DOLLY) RDU:G1 14 RVY:7B RR7:11 WARD RKB:3 UNIT IFU 301 3 RESP MSS 32M STARLING RWJ:A11 YELLOW YELLOW RWW:A1 RWW:A2 RVY:11B RWP:A5 RKB:2 R1H:11F RWH:9B ENDO RR7:4 30M F7) 5M 17 12 45 7 A9 2 RWP:AMAUM RWH:MAU RWJ:A14 RKE:ITU RXR:C4 VM0C0:UNIT RWY:WARD RJ2:QEW RJE:WARD RXC:WELLINGTON RH5:CREWKERNE RXC:NEWINGTON RLQ:WYE R1G:TAVISTOCK RNL:WILLOW RMP:WARD RP5:WARD RDE:DEDHAM RJ6:DUPPAS RNZ:REDLYNCH RBD:ABBOTSBURY RWD:GRANTHAM RJE:WARD RWG:AAU RA7:BRI RLQ:ARROW RTD:RVI RWY:WARD RNQ:CRANFORD ONE RH5:LYDEARD RBK:WARD RBT:WARD RP5:WARD RBL:WARD RJF:WARD RXF:38 RDZ:RB01 LOWSON RLN:F61 INVICTA 2 RGR:F9 RGT:N2 RGT:N3 10 RFS:20 L37 6 RFS:21 WARD EAST RXR:C8 RGT:D5 RGT:F6 RTE:7B RTE:6A RVY:7A RXW:7 WARD SMH 312 3 MSSU CDU WORTHING RHM:AMU3 RGT:EAU4 RWH:AAU RNQ:CLIFFORD RLQ:ROSS R1G:NEWTON RGR:F7 RAJ:GORDON RTP:GODSTONE RVR:EGH RVR:EGH COMMUNITY AMU RWF:CHAUCER RLT:MELLY RFW:OSTERLEY RA7:BHI RTD:FH RLT:FELIX RNL:HONISTER RWF:WARD AND RTX:FGH RVJ:WARD RMP:WARD RMP:WARD RC3:6 RDD:ORSETT RQW:LOCKE RXP:44 RXP:14 AND RBD:EMU RXR:MAU RC3:AMU RN7:OAK RVY:14B RFW:AMU RCF:AMU RTP:AMU 123 RM2:F14 RL4:C16 RXR:C6 RGT:C4 RWJ:A1 RVY:9B WARD 10 18 46 1 RR7:1 UNIT PGH A5 40 WARD RP5:WARD RC9:WARD RD8:WARD RBT:WARD ABBOT RVR:EGH RXF:QUEEN RWD:LINCOLN RNQ:HARROWDEN RQM:EDGAR RTP:HAZELWOOD RD1:CHESELDEN RTD:FH REF:GRENVILLE RWH:DIGSWELL RH5:BURNHAM CHUTER RCX:TERRINGTON RTD:FH REF:MED BUCKLEY RN3:NEPTUNE RDE:NAYLAND RA3:KEWSTOKE RJE:WARD R1H:PLASHET RA9:SIMPSON RWG:WHITE RYR:ASHLING RCF:WARD RC9:WARD CROFT RK5:SCONCE RQW:LISTER SHORT RDD:WHMB RWP:WASU RJ2:QEW ELIZABETH RTE:RYEWORTH WARD RCX:PENTNEY RWP:WAVON3 RBK:WARD RNL:LARCH RP5:WARD RBT:WARD RNL:JENKIN RXC:BAIRD HOSPITAL TEMPLAR RQW:RAY 218 RCX:MAU HOPKINS RGN:A10 RGR:F10 RTG:306 L3 SOUTH RGR:G5 RXR:C3 RJ6:AMU RL4:C19 RDU:F3 RTE:7A RTE:8B RTE:6B RFF:28 (122) TWO CRH BLUE GASTRO RXK:D41 RM2:F12 RDU:F9 RFR:B1 RKB:20 RFS:14 RR7:24 RR7:25 6 DMH 15B HOLT (123) HRI WARD 107 A4 54 41 42 44 17 14 24 26 B 12 18 21 31 2 RHM:AMU2 RDD:EFMB RXP:52 RBT:WARD RXP:1 RWP:WAMU RDD:ECMB RXP:6 REM:MAU EDE RLN:B28 RLN:E53 RBN:2A RXR:C9 HORNE 2 WARD STAY 61 8 17 15 11 RRF:INCE BURTON RWW:A8 RGR:G4 HOUSE RFR:A5 RTE:4B RFS:16 DIXON UHND RFS:5 RFS:1 WCH DMH AMU 210 BAH 03 26 16 CA 2 15A7 (Intercept) 2 0 2 4 Prof. Dr. Michael Simon 32
Example Germany PTVS Prof. Dr. Michael Simon 33
Background >13,000 Nursing homes in Germany Since 2009 annual inspections of the Medical Advisory Services of the Statutory Health Insurance 82 pre-defined criteria and summarised by grades (PTVS) Critique: Emphasis on structures and processes and its limited ability to differentiate between nursing homes. Prof. Dr. Michael Simon 34
PTVS data Care transparency agreement home care Overall Quality Rating -- Medical & Nursing Care Dementia Care Social environment and daily activities Room and board services -- Resident interviews Prof. Dr. Michael Simon 35
3Q-Study Internal Quality Management Monitoring of a notfor-profit organisation in Germany 83 Nursing homes Annual Survey of staff % of residents with a fall Prof. Dr. Michael Simon 36
Overall Quality better Nurse perceived quality PTVS Overall Rating better Prof. Dr. Michael Simon 37
Risk assessment (measured by PTVS) vs. % residents with fall better better Prof. Dr. Michael Simon 38
Fall prevention (measured by PTVS) vs. % residents with fall better better Prof. Dr. Michael Simon 39
Falls vs. different criteria from PTVS!! %! O/E! r! p! r! p! (QGS)! Overall!Score! 0,04! 0,60! 0,09! 0,25! (QB1)! Medical!and!nursing!care! 0,01! 0,88! D0,01! 0,92! Falls! (T24)! Fall!risk!assessment! D0,03! 0,75! 0,01! 0,90! (T25)! Documentation! D0,03! 0,73! 0,01! 0,90! (T26)! Fall!prevention! D0,06! 0,46! D0,02! 0,81!! Prof. Dr. Michael Simon 40
Summary The good, the bad and the ugly Educational needs Research on indicators Research on implementation Health policy Prof. Dr. Michael Simon 41
Literatur Berwick, D. M., B. James and M. J. Coye (2003). "Connections between quality measurement and improvement." Med Care 41(1 Suppl): I30-38. Bouldin, E. L., E. M. Andresen, N. E. Dunton, M. Simon, T. M. Waters, M. Liu, M. J. Daniels, L. C. Mion and R. I. Shorr (2013). "Falls Among Adult Patients Hospitalized in the United States: Prevalence and Trends." Journal of Patient Safety 9(1): 13-17. Dixon-Woods, M., R. Baker, K. Charles, J. Dawson, G. Jerzembek, G. Martin, I. McCarthy, L. McKee, J. Minion, P. Ozieranski, J. Willars, P. Wilkie and M. West (2013). "Culture and behaviour in the English National Health Service: overview of lessons from a large multimethod study." BMJ Quality & Safety. Donabedian, A. (1988). "The quality of care. How can it be assessed?" JAMA : the journal of the American Medical Association 260(12): 1743-1748. Donabedian, A. (1992). "The role of outcomes in quality assessment and assurance." QRB Qual Rev Bull 18(11): 356-360. Normand, S.-L. T., M. E. Glickman and C. A. Gatsonis (1997). "Statistical Methods for Profiling Providers of Medical Care: Issues and Applications." Journal of the American Statistical Association 92(439): 803-814. Normand, S.-L. T. and D. M. Shahian (2007). "Statistical and Clinical Aspects of Hospital Outcomes Profiling." Statistical Science 22(2): 206-226. Simon, M. and N. Dunton (2014). Entwicklung, Erprobung und Anwendung von Qualitätsindikatoren der Pflege im Krankenhaus: das Beispiel NDNQI aus den USA. Qualitätsentwicklung in der Pflege - Methoden und Instrumente. D. Schiemann, M. Moers and A. Büscher, Kohlhammer. Simon, M., S. Klaus, B. J. Gajewski and N. Dunton (2013). "Agreement of Fall Classifications Among Staff in U.S. Hospitals." Nursing Research 62(2): 74-81. Simon, M., S. Schmidt, C. G. G. Schwab, H.-M. Hasselhorn and S. Bartholomeyczik (2013). "Messung der Pflegequalität in der Langzeitpflege - Eine vergleichende Analyse von Pflegetransparenzkriterien, bewohnerbezogenen Indikatoren und Beurteilungen der Mitarbeiter." Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz 56(8): 1088-1097. van Dishoeck, A. M., H. F. Lingsma, J. P. Mackenbach and E. W. Steyerberg (2011). "Random variation and rankability of hospitals using outcome indicators." BMJ Qual Saf 20(10): 869-874. Prof. Dr. Michael Simon 42