HCAHPS Update Training

HCAHPS Update Training

Welcome! In the Update Training sessions, we will present: HCAHPS Program Updates Updates on HCAHPS Quality Assurance Guidelines V 6.0 Calculation of HCAHPS Scores: From Raw Data to Publicly Reported Results Exploring the HCAHPS Summary Tables and Charts Key Findings From Published HCAHPS Research HCAHPS and Proposed Hospital Value-Based Purchasing (HVBP) 2

Program Updates 3

Overview Hospital Compare Updates Changes to the Quality Assurance Guidelines V 6.0 New charts that include HCAHPS Hospital Characteristics Comparisons available on www.hcahpsonline.org New Research article 4

Hospital Compare: New Look and Feel 5

Hospital Compare: New Look and Feel (cont d) 6

Hospital Compare: New Hospital Profile Page 7

Timing of Hospital Compare Refreshes Anticipated Public Reporting Dates: April 2011 (patient discharges from July 2009 June 2010) July 2011 (patient discharges from October 2009 September 2010) October 2011 (patient discharges from January 2010 December 2010) January 2012 (patient discharges from April 2010 March 2011) 8

Revised Footnotes for July 2011 Public Reporting New Footnote 12 Very few patients were eligible for the HCAHPS survey. The scores shown reflect fewer than 50 completed surveys. Use these scores with caution, as the number of surveys may be too low to reliably assess hospital performance Revised Footnote 9 No or very few patients were eligible for the HCAHPS survey 9

Updates to QAG New exclusion category Patients discharged to Nursing Homes or Skilled Nursing Facilities Lag time required for all HCAHPS Final Survey Status codes Number of survey attempts is now a required field Changes to Question 27 in the HCAHPS Survey Minor telephone script changes 10

HCAHPS Hospital Characteristics Comparison Charts Introduced in December 2010 on HCAHPS Executive Insight Presents hospital performance on the ten publicly reported HCAHPS measures by: Region Bed Size Teaching Status Ownership and Control Location 11

New HCAHPS Research Hospital Survey Shows Improvements in Patient Experience. M.N. Elliott, W.G. Lehrman, E.H. Goldstein, L.A.Giordano, M.K. Beckett, C.W. Cohea and P.D. Cleary. Health Affairs, 29 (11): 2061-2067. 2010. 12

Updates HCAHPS Quality Assurance Guidelines (QAG) V6.0 13

Updates Survey Management Customer support line Telephone staffed live during core business hours Voice mail is acceptable after hours, but must be regularly monitored and replied to within one business day Voice mail recording must specify that the caller can leave a message about the survey Hospitals/Survey vendors must document questions and responses Safeguarding Patient Confidentiality Establish protocols for secure file transmission Emailing of PHI via unsecure email is prohibited 14

Survey Management (cont d) Clarifications Data Storage All HCAHPS-related data files, including patient discharge files, must be retained for a minimum of three years All data files must be easily retrievable 15

Updates Sampling Sampling section content has been reorganized New sampling examples have been added The lower precision of scores based on less than 100 and less than 50 completed surveys will be noted in public reporting 16

Sampling (cont d) Update New Exclusion Category Patients discharged to nursing homes and skilled nursing facilities (beginning with July 1, 2011 and forward patient discharges) This applies only to patients with a Discharge Status (UB-04 field location 17) of: 03 (Skilled nursing facility) 64 (Certified Medicaid nursing facility) 17

Sampling (cont d) Updates MS-DRG Codes Updated to MS-DRG V.28 Strongly recommend use of MS-DRG codes to assign Service Line Validate that Service Line is assigned appropriately 18

Clarifications Sampling (cont d) Patients may voluntarily sign a no publicity request while in the hospital Sampling rates should be consistent among the months in a given quarter PSRS and DSRS methodologies further defined 19

Survey Administration Updates All modes of survey administration: Mail Only, Telephone Only, Mixed Mode, Active IVR Q27 (Language Speak): Chinese, Russian & Vietnamese response options added Lag time must be recorded for all HCAHPS Final Survey Status codes Interim Disposition Codes Hospitals/Survey vendors must maintain and provide a crosswalk of their interim disposition codes to the HCAHPS Final Survey Status codes 20

Survey Administration (cont d) Clarifications All modes of survey administration: Mail Only, Telephone Only, Mixed Mode, Active IVR Cannot show the HCAHPS survey or cover letter to patients prior to discharge from the hospital Do not mail any pre-notification letters or postcards informing patient about the HCAHPS survey Supplemental questions cannot ask the patient to explain why he or she chose their specific response Example: It is not acceptable to ask patients why they would or would not recommend the hospital to friends and family 21

Survey Administration Update Copyright language In the event an organization copyrights their survey materials, then the following language must be included: The Core HCAHPS questions (Questions 1-22) and About You HCAHPS questions (Questions 23-27) are works of the U.S. Government. These HCAHPS questions are in the public domain and therefore are NOT subject to U.S. copyright laws. 22

Survey Administration (cont d) Update Mail Only New Field: Number Survey Attempts Mail Capture the mail wave in which the final disposition of the survey is determined 23

Survey Administration (cont d) Update Mail Only Chinese instrument Added underlining to Q3, Q5, Q6, and Q7 24

Survey Administration (cont d) Update Mail Only (cont d) Vietnamese instrument Q27. Quý vị dùng ngôn ngữ nào chính trong nhà? 25

Survey Administration (cont d) Update Mail Only & Mixed Mode Spanish instrument Intro before Q1 Las siguientes preguntas se refieren sólo a la vez que estuvo en el hospital cuyo nombre aparece en la carta de presentación de esta encuesta. No incluya información sobre otras veces que estuvo en otro hospital. 26

Survey Administration (cont d) Update Mail Only & Mixed Mode Spanish instrument (cont d) Intro before Q21 Por favor conteste las siguientes preguntas sobre la vez que estuvo en el hospital cuyo nombre aparece en la carta de presentación de esta encuesta. No incluya información sobre otras veces que estuvo en otro hospital. 27

Survey Administration (cont d) Clarifications Mail Only & Mixed Mode Cover letter survey responses will be shared with the hospital for quality improvement purposes (add this phrase, if applicable) Seeded mailings must be part of the survey process 28

Survey Administration (cont d) Clarifications Mail Only & Mixed Mode (cont d) Guidelines for patients who request an additional survey to be sent Mail Only: acceptable to mail a replacement survey at the patient s request within the 42-day survey administration period Mixed Mode: acceptable to mail a replacement survey at the patient s request within the first 21 days of the 42-day survey administration period. After 21 days from the mailing, a replacement HCAHPS survey must NOT be mailed-out as the telephone portion of the mixed mode protocol must be initiated 29

Survey Administration (cont d) Updates Telephone Only, Mixed Mode, and Active IVR New Field: Number Survey Attempts Telephone Capture the telephone attempt in which the final disposition of the survey is determined Caller ID may be programmed to display on behalf of [HOSPITAL NAME] (with permission of hospital s HIPAA compliance officer) 30

Survey Administration (cont d) Updates Telephone Only, Mixed Mode, and Active IVR (cont d) Calls that are inadvertently dropped must be re-contacted immediately Do not program a specific response category as a default option Guidelines for monitoring/recording telephone calls in QAG, Pages 75, 94 and 105 31

Survey Administration (cont d) Updates Telephone Only, Mixed Mode, and Active IVR (cont d) Telephone script Initiating Contact Updated response to IF ASKED WHO IS CALLING Q26 Added I am required to read all five categories. Q27 Added programming and additional language response options (i.e., Chinese, Russian & Vietnamese) 32

Survey Administration (cont d) Clarifications Telephone Only, Mixed Mode, and Active IVR All underlined content must be emphasized and no other script content is to be emphasized Reaching an answering machine/voicemail is considered an attempt If it is determined that the patient may be available in the latter part of the 42-day data collection time period, then hospitals/survey vendors must use the entire data collection time period to schedule telephone calls Strongly encouraged to provide visual oversight of the interviewing staff in addition to required silent monitoring 33

Data Specifications & Coding Updates File Specifications Version 3.3 (beginning with July 1, 2011 and forward patient discharges) New Patient Administrative Data Fields Number Survey Attempts Telephone Number Survey Attempts Mail Update Lag Time assignments for all HCAHPS Final Survey Status codes 34

Data Specifications & Coding (cont d) Update Number Survey Attempts Telephone Required when: Survey Mode is 2 Telephone Only or 4 - IVR Survey Mode is 3 Mixed mode and Survey Completion Mode is 2 Mixed mode-phone Not Required when: Survey Mode is 1 Mail Only If this field ( Survey Attempts Telephone ) is included with Survey Mode of 1 Mail Only, then code Survey Attempts Telephone, as 8 Not applicable 35

Data Specifications & Coding (cont d) Update Number Survey Attempts Mail Required when: Survey Mode is 1 Mail Only Not Required when: Survey Mode is 2 Telephone Only, 3 Mixed mode, or 4 IVR If this field ( Survey Attempts Mail ) is included with Survey Mode of 2 Telephone Only, 3 Mixed mode, or 4 IVR, then code Survey Attempts Mail, as 8 Not applicable 36

Data Specifications & Coding (cont d) Update Lag Time Calculated for each patient in the sample Defined as the number of days between the patient s discharge date from the hospital and the date that data collection activities ended for the patient All HCAHPS Final Survey Status codes for each patient in the sample must contain the actual lag time Do NOT use code 888 Not Applicable 37

Data Specifications & Coding Clarifications (cont d) Zero cases and five or fewer eligible HCAHPS patient discharges for data submission Must not be used for hospitals/survey vendors that missed surveying eligible patients for the entire month (e.g., when hospitals do not submit the entire month discharge list to their survey vendor in a timely manner, etc.) Do not submit a Header Record, rather a Discrepancy Report is submitted online Enter all of the race categories that the patient has selected 38

Data Specifications & Coding Clarifications (cont d) If screener questions are not answered and coded as M Missing/Don t Know, then the dependent questions that are not answered must also be coded as M Missing/Don t Know 12. During this hospital stay, did you need medicine for pain? Yes No If no, Go to Question 15 13. During this hospital stay, how often was your pain well controlled? Never Sometimes Usually Always Example Code as: M - Missing/Don t Know Code as: M - Missing/Don t Know 39

Updates Oversight Activities Quality Assurance Plan Free of extraneous information Concise explanations of required HCAHPS processes Include a crosswalk of interim disposition codes to HCAHPS Final Survey Status codes Due date April 15, 2011 On-site Visits Project Director/Project Manager must be physically present during the on-site visit 40

Questions? 41

Calculation of HCAHPS Scores: From Raw Data to Publicly Reported Results 42

Overview of Presentation Process Submitted HCAHPS Data Calculate HCAHPS Scores w/example HCAHPS Composite Measure Calculation Patient-Mix Variables Final HCAHPS scores 43

Process Submitted HCAHPS Data Remove surveys determined to be ineligible for HCAHPS Exclude surveys that do not meet HCAHPS criteria for survey completeness HCAHPS scores are calculated from eligible and completed surveys 44

HCAHPS Boxes Top-Box : most positive response category Always for 5 HCAHPS Composites, Yes for Discharge Information, 9 or 10 for Hospital Rating, and Definitely for Recommend the Hospital Middle-Box : in-between response category Usually for 5 HCAHPS Composites, does not exist for Discharge Information, 7 or 8 for Hospital Rating, and Probably Yes for Recommend the Hospital Bottom-Box : least positive response category Sometimes or Never for 5 HCAHPS Composites, No for Discharge Information, 0 through 6 for Hospital Rating, and Definitely No or Probably No for Recommend the Hospital 45

HCAHPS Composite Example Nurse Communication, Top-Box Always is the most positive response category for all 3 questions that comprise Nurse Communication Q1: Nurse courtesy and respect Q2: Nurse listen Q3: Nurse explain 46

HCAHPS Composite Example (cont d) 1. During this hospital stay, how often did nurses treat you with courtesy and respect? 1 Never 2 Sometimes 3 Usually 4 Always 47

HCAHPS Composite Example (cont d) Hospital A has 7 completed surveys Survey ID Q1 Response Nurse1 Top- Box Q2 Response Nurse2 Top- Box Q3 Response Nurse3 Top- Box 1 Always 1 Always 1 Always 1 2 Sometimes 0 Never 0 Sometimes 0 3 Always 1 Always 1 Always 1 4 Usually 0 Always 1 Always 1 5 Always 1 Always 1 Always 1 6 Always 1 Usually 0 Always 1 7 Always 1 Always 1 Always 1 48

HCAHPS Composite Example (cont d) Calculate mean for each of Nurse1 Top-Box, Nurse2 Top-Box, and Nurse3 Top-Box: Nurse1 Top-Box mean = (1+0+1+0+1+1+1) / 7 = 5/7 Nurse2 Top-Box mean = (1+0+1+1+1+0+1) / 7 = 5/7 Nurse3 Top-Box mean = (1+0+1+1+1+1+1) / 7 = 6/7 49

HCAHPS Composite Example (cont d) Nurse Communication composite score for Hospital A Calculate the Nurse Communication composite mean (Y) as follows: Y = (Nurse1 Top-Box mean + Nurse2 Top-Box mean + Nurse3 Top-Box mean) / 3 = (5/7 + 5/7 +6/7) / 3 = 0.762 50

Patient-Mix Variables: Education 24. What is the highest grade or level of school that you have completed? 1 8 th grade or less 2 Some high school, but did not graduate 3 High school graduate or GED 4 Some college or 2-year degree 5 4-year college graduate 6 More than 4-year college degree 51

Patient-Mix Variables: Education (cont d) Question 24 for Hospital A Education: Derived from HCAHPS survey (values range from 1 to 6) Calculate the Education mean for Hospital A: H EDUC = (4+1+ +3) / 7 = 22/7 = 3.14 Survey ID Education 1 4 2 1 3 6 4 5 5 1 6 2 7 3 52

Patient-Mix Variables: Overall Health 23. In general, how would you rate your overall health? 1 Excellent 2 Very Good 3 Good 4 Fair 5 Poor 53

Patient-Mix Variables: Overall Health (cont d) Question 23 for Hospital A Self-Rated Health: Derived from HCAHPS survey (values range from 1 to 5) Calculate the Self-Rated Health mean for Hospital A: H HLTH = (5+5+ +4) / 7 = 24/7 = 3.43 Survey ID Self-Rated Health 1 5 2 5 3 2 4 3 5 4 6 1 7 4 54

Patient-Mix Variables: Non-English Language 27. What language do you mainly speak at home? 1 English 2 Spanish 3 Some other language (please print) 55

Patient-Mix Variables: Non-English Language (cont d) Question 27 for Hospital A Non-English Language If Q27 answer value is Spanish or Some other language then let indicator variable Non-English = 1, otherwise Non-English = 0 Calculate the Non-English mean for Hospital A: H NENG = (0+1+ +0) / 7 = 2/7 = 0.29 Survey ID Question 27 Non- English 1 English 0 2 Spanish 1 3 Some other 1 4 English 0 5 English 0 6 English 0 7 English 0 56

Patient-Mix Variables: Age Range Age Ranges for Hospital A Create 7 indicator (0 or 1) age variables for each of the 8 age range groups (Age 85+ is the reference category): Survey ID Age 18-24 Age 25-34 Age 35-44 Age 45-54 Age 55-64 Age 65-74 Age 75-84 1 0 0 0 0 0 1 0 2 0 0 0 0 1 0 0 3 0 1 0 0 0 0 0 4 0 0 0 0 0 0 0 5 1 0 0 0 0 0 0 6 0 0 1 0 0 0 0 7 0 0 0 0 0 1 0 57

Patient-Mix Variables: Age Range (cont d) For Hospital A, calculate mean of each of the 7 age range variables H 18-24 = (0+0+0+0+1+0+0) / 7 = 1/7 = 0.14 H 25-34 = (0+0+1+0+0+0+0) / 7 = 1/7 = 0.14 H 35-44 = (0+0+0+0+0+1+0) / 7 = 1/7 = 0.14 H 45-54 = (0+0+0+0+0+0+0) / 7 = 0/7 = 0.00 H 55-64 = (0+1+0+0+0+0+0) / 7 = 1/7 = 0.14 H 65-74 = (1+0+0+0+0+0+1) / 7 = 2/7 = 0.29 H 75-84 = (0+0+0+0+0+0+0) / 7 = 0/7 = 0.00 58

Patient-Mix Variables: Service Line Hospital A s Service Line Create an indicator (0 or 1) variable for Maternity and Surgical (Medical is reference category): Survey ID Service Line Maternity Surgical 1 Surgical 0 1 2 Medical 0 0 3 Maternity 1 0 4 Medical 0 0 5 Maternity 1 0 6 Medical 0 0 7 Medical 0 0 59

Patient-Mix Variables: Service Line (cont d) Calculate Hospital A s mean for Maternity and Surgical variables H MAT = (0+0+1+0+1+0+0) / 7 = 2/7 = 0.29 H SURG = (1+0+0+0+0+0+0) / 7 = 1/7 = 0.14 60

Patient-Mix Variables: Service Line x Age Service Line x Age Interaction Variables Create single variable called Age that takes on values 1 through 8, to indicate a patient s age range Age Range: 18-24 25-34 35-44 45-54 55-64 65-74 75-84 85+ Age: 1 2 3 4 5 6 7 8 61

Patient-Mix Variables: Service Line x Age (cont d) Use variables Surgical, Maternity, and Age to create the following interaction variables: Surgical*Age = (Surgical) x (Age) Maternity*Age = (Maternity) x (Age) 62

Patient-Mix Variables: Service Line x Age (cont d) Service Line x Age Interaction for Hospital A Survey ID Age Maternity Surgical Maternity* Age Surgical* Age 1 6 0 1 0*6 = 0 1*6 = 6 2 5 0 0 0*5 = 0 0*5 = 0 3 2 1 0 1*2 = 2 0*2 = 0 4 8 0 0 0*8 = 0 0*8 = 0 5 1 1 0 1*1 = 1 0*8 = 0 6 3 0 0 0*3 = 0 0*3 = 0 7 6 0 0 0*6 = 0 0*6 = 0 63

Patient-Mix Variables: Service Line x Age (cont d) For Hospital A, calculate the mean for Maternity x Age and Surgical x Age variables: H MAT*AGE = (0+0+2+0+1+0+0) / 7 = 3/7 = 0.43 H SURG*AGE = (6+0+0+0+0+0+0) / 7 = 6/7 = 0.86 64

Patient-Mix Variables: Response Percentile

Patient-Mix Variables: Response Percentile (cont d) Response Percentile for Hospital A Response Percentile = Lag Time Rank / Monthly Sample Size Survey ID Month Lag Time (in days) Lag Time Rank (by month) Monthly Sample Size Response Percentile 1 January 21 1 9 1/9 2 January 34 2 9 2/9 3 February 8 1 11 1/11 4 February 12 2 11 2/11 5 March 29 2 8 2/8 6 March 6 1 8 1/8 7 March 30 3 8 3/8

Patient-Mix Variables: Response Percentile (cont d) Calculate Hospital A s mean Response Percentile for the quarter H RPCT = [(1/9)+(2/9)+(1/11)+(2/11)+(2/8) +(1/8)+(3/8)] / 7 = 0.19 67

3 Components needed for Hospital PMA Hospital means for each patient-mix variable (just calculated) National means for each patient mix variable found on www.hcahpsonline.org Patient-level Adjustments for each patient-mix variable found on www.hcahpsonline.org 68

PMA Means for Hospital A H EDUC = 3.14 H 18-24 = 0.14 H MAT = 0.29 H 25-34 = 0.14 H SURG = 0.14 H HLTH = 3.43 H 35-44 = 0.14 H 45-54 = 0.00 H MAT*AGE = 0.43 H NENG = 0.29 H 55-64 = 0.14 H SURG*AGE = 0.86 H 65-74 = 0.29 H RPCT = 0.19 H 75-84 = 0.00 69

National Means for Patient-Mix Variables: Table 3 from www.hcahpsonline.org 70

National Means for Patient-Mix Variables (from Table 3) (cont d) M EDUC = 3.65 M 18-24 = 0.05 M MAT = 0.15 M 25-34 = 0.11 M SURG = 0.34 M HLTH = 2.82 M 35-44 = 0.08 M 45-54 = 0.12 M MAT*AGE = 0.38 M NENG = 0.06 M 55-64 = 0.18 M SURG*AGE = 1.81 M 65-74 = 0.21 M RPCT = 0.16 M 75-84 = 0.19 71

Patient-Level Adjustments for Patient-Mix Variables: Table 1 from www.hcahpsonline.org 72

Patient-Level Adjustments for Patient- Mix Variables (from Table 1) (cont d) For Nurse Communication: A EDUC = 2.12% A 18-24 = 4.18% A MAT = -6.27% A HLTH = 5.28% A 35-44 = -1.08% A 25-34 = 1.21% A SURG = 1.64% A 45-54 = -3.24% A MAT*AGE = 1.06% A NENG = -0.04% A 55-64 = -5.14% A SURG*AGE = -0.07% A 65-74 = -5.06% A RPCT = 0.20% A 75-84 = -3.04% 73

Using PMA Equation PMA Equation: PMA = A EDUC *(H EDUC M EDUC ) + A HLTH *(H HLTH M HLTH ) + A NENG *(H NENG M NENG ) + A RPCT *(H RPCT - M RPCT ) + A 1824 *(H 1824 M 1824 ) + A 2534 *(H 2534 M 2534 ) + A 3544 *(H 3544 M 3544 ) + A 4554 *(H 4554 M 4554 ) + A 5564 *(H 5564 M 5564 ) + A 6574 *(H 6574 M 6574 ) + A 7584 *(H 7584 M 7584 ) + A MAT *(H MAT M MAT ) + A SURG *(H SURG M SURG ) + A MAT*AGE *(H MAT*AGE M MAT*AGE ) + A SURG*AGE *(H SURG*AGE M SURG*AGE ) Net Adjustment for Hospital A: PMA = 0.021*(3.14 3.65) + 0.053*(3.43 2.82) + -0.000*(0.29 0.06) + 0.002*(0.19 0.16) + 0.042*(0.14 0.05) + 0.012*(0.14 0.11) + -0.011*(0.14 0.08) + -0.032*(0.00 0.12) + -0.051*(0.14 0.18) + -0.051*(0.29 0.21) + -0.030*(0.00 0.19) + -0.063*(0.29 0.15) + 0.016*(0.14 0.34) + 0.011*(0.43 0.38) + -0.001*(0.86 1.81) = 0.022 74

Survey Mode Adjustment Mode Adjustment for Nurse Communication Top-Box : Hospital A utilizes the Telephone mode for HCAHPS HCAHPS Composite or Item: Top-Box Communication with Nurses ( Always ) Communication with Doctors ( Always ) Telephone Mixed Active IVR -4.0% -0.3% -1.8% -1.3% 1.0% -0.3% Hospital Rating ( 9 or 10 ) -2.8% -1.8% -1.6% 75

Quarterly HCAHPS Adjusted Score The sum of the unadjusted composite score (Y), hospital PMA, and survey mode adjustment comprise a hospital s quarterly adjusted HCAHPS score (Y ) Nurse Communication Top-Box for Hospital A: Y = Y + PMA + Mode Adjustment = 0.762 + 0.022 + -0.040 = 0.744 (or 74.4%) 76

Publicly Reported HCAHPS Scores Publicly reported HCAHPS scores are the average of the adjusted scores from 4 consecutive quarters For Hospital A: Adjusted Top-Box scores for Nurse Communication for 3 previous quarters equal 71.5%, 78.8%, and 79.9% Thus, the rounded publicly reported Top-Box score for Nurse Communication is as follows: (71.5% + 78.8% + 79.9% + 74.4%) / 4 = 76% 77

Publicly Reported HCAHPS Scores (cont d) Nurse Communication for Hospital A Adjusted Top-Box Adjusted Middle-Box Adjusted Bottom- Box Quarter 1 71.5% 21.6% 6.9% Quarter 2 78.8% 16.4% 4.8% Quarter 3 79.9% 15.7% 4.4% Quarter 4 74.4% 19.9% 5.7% Q1-Q4 Avg. 76.2% 18.3% 5.5% Rounded* 76% 18% 6% *HCAHPS score publicly reported on Hospital Compare 78

Review of Score Calculation Identify eligible and completed surveys Calculate un-adjusted quarterly scores for each HCAHPS measure Calculate hospital-level patient-mix adjustment (PMA) Requires calculation of means for all patient-mix variables and utilization of HCAHPS PMA Tables Application of survey mode adjustment Publicly reported HCAHPS scores = Average of the adjusted scores from 4 consecutive quarters 79

Questions? 80

BREAK 81

Exploring HCAHPS Summary Tables & Charts 82

Overview Walk through Tables and Charts included on www.hcahpsonline.org Summary of Survey Results National and State means HCAHPS Percentiles Top-box and Bottom-box Comparison of Scores by Hospital Characteristics New Charts Correlations among HCAHPS Measures 83

Summary Analyses on www.hcahpsonline.org 84

Summary of Survey Results Summary of HCAHPS Survey Results Table Top-box scores Roll-up of four quarters Current table from www.hcahpsonline.org Updated quarterly Includes Archive of past Summary tables 85

86

Summary of HCAHPS Survey Results April 2009 to March 2010 87

Top-box and Bottom-box Percentiles Table Drill down further to understand better Your hospital s HCAHPS scores: Both Top-box and Bottom-box How much above/below other hospitals? 88

HCAHPS Percentiles December 2010 Public Reporting 89

HCAHPS Percentiles: Top-box Most positive survey response The higher, the better Used in Summary Table, etc. 90

HCAHPS Percentiles: Bottom-box Least positive survey response How often patients responded negatively The lower, the better 91

Comparison of Scores by Hospital Characteristics HCAHPS Hospital Characteristics Comparison Charts Newest addition to Summary Analyses page Region (Census division) Location (Urban/Rural) Bed Size Teaching Status Ownership and Control 92

Communication with Nurses Patients discharged between July 2008 and June 2009 93

Communication with Nurses Region Patients discharged between July 2008 and June 2009 94

Communication with Nurses Location Patients discharged between July 2008 and June 2009 95

Communication with Nurses Bed Size Patients discharged between July 2008 and June 2009 96

Communication with Nurses Teaching Status Patients discharged between July 2008 and June 2009 97

Communication with Nurses Ownership & Control Patients discharged between July 2008 and June 2009 98

HCAHPS Correlations Table Drill down further to understand better How are HCAHPS measures related to each other? 99

HCAHPS Correlations July 2008 June 2009 Discharges 100

Questions? 101

Key Findings From HCAHPS Research 102

Overview of Research Reliability of HCAHPS Measures Improvement in HCAHPS Scores Hospital-Level Factors Associated with HCAHPS Scores Racial/Ethnic Variation in HCAHPS Scores 103

Reliability of HCAHPS Measures [see Elliott et al. Do Hospitals Rank Differently on HCAHPS for Different Patient Subgroups? Medical Care Research and Review 2010 Feb;67(1):56-73.] 104

Objective Describe the reliability of reported HCAHPS measures at the hospital level-is hospital performance measured well at the recommended sample sizes? Here reliability is the proportion of the variance in reported hospital scores that is due to true variation between hospitals, rather than within-hospital variation that reflects limited sample size. At 0.0 reliability, there are no true differences between hospitals If hospitals truly differed and sample sizes were infinite, reliability would be 1.0 At reliability 0.7, 70% of the variance of reported hospital scores would be attributable to true variation in hospital performancethe part we d see with limitless sample sizes 105

Data and Measures Used all 10 HCAHPS measures but combined Cleanliness & Quietness Data from 1.2 million patients discharged October 2006-June 2007 from 2,684 hospitals in March 2008 public reporting 106

Methods HCAHPS measures are adjusted for survey mode MIXED linear models of HCAHPS measures Hospital-level variance components are random effects Patient-mix adjustment variables are fixed effects These models assess the hospital-level reliability of reported scores 107

Reported Hospital Scores are Highly Reliable at Recommended Sample Sizes Large hospital-level variance components produced hospital-level reliabilities of adjusted scores at n=300 survey completes per item At least 0.92 for 5/9 HCAHPS measures (Overall Rating, Recommendation, Nurse Communication, Staff Responsiveness and Clean/Quiet) At least 0.82 for all HCAHPS measures 108

Reported Hospital Scores are Adequately Reliable at Smaller Sample Sizes At 100 survey completes, reliabilities are at least 0.70 for 7/9 HCAHPS measures Exceptions are Communication about Medicines (0.67) and Pain Management (0.61) On average, these measures are applicable to only 72% and 61% of patients, respectively 109

Improvement in HCAHPS Scores over the First Two Years of Public Reporting [See Elliott et al., Hospital survey shows improvements in patient experience. Health Affairs 2010 Nov;29(11):2061-7] 110

Objective Assess the extent and uniformity of improvement in HCAHPS scores in the first 2 years of public reporting among initially participating hospitals 111

Study Design The study was designed to examine changes in HCAHPS scores in the year following initial public reporting among original participants HCAHPS results were first publicly reported in March 2008 Discharges from October 2006 - June 2007 The 5 th quarterly public reporting update in March 2009 Discharges from July 2007 - June 2008 Mutually exclusive of March 2008 112

Data Data come from 3.4 million patients from the 2,774 hospitals that collected data and qualified for both March 2008 and March 2009 public reporting Here we do not consider data from the >1000 hospitals added after 3/08 public reporting 113

Approach Used 9 measures- all but Rating of Hospital Analyses performed for this research are based on top box (most positive response option) hospital scores 114

Top Box Means of Hospitals Participating in Both March 2008 & March 2009 (*p<0.01) March 2008 March 2009 Difference Nurse Communication 72.7 73.1 +0.4* Doctor Communication 79.1 79.0-0.1 Responsiveness of Hosp. Staff 59.9 60.8 +0.9* Pain Management 67.1 67.5 +0.4* Communication about Meds. 57.5 58.0 +0.5* Cleanliness 67.9 68.3 +0.4* Quietness 53.6 54.5 +0.8* Discharge Information 79.1 79.9 +0.8* Willingness to Recommend 67.1 67.4 +0.3* 115

Improvement in HCAHPS Measures (3/08 to 3/09) Significant improvement in 8 of 9 measures (p<0.01) Exception was Doctor Communication (p=0.25) Magnitude of improvement was relatively uniform across items Total improvement did not vary significantly by hospital size, region, or ownership 116

Most and Least Improved HCAHPS Measures Greatest improvements for Discharge Information and Staff Responsiveness Discharge Information improved by 5 percentile points of rank (e.g. going from the 50 th to 55 th percentile) Staff Responsiveness improved by 4 percentile points of rank (e.g. going from the 50 th to 54 th percentile) Smallest gain for Hospital Recommendation (1 percentile point increase) No significant change in Doctor Communication 117

Observations on Most Improved Measures Improvement seen in concrete, actionable domains E.g., Quietness and Discharge Information These items may have been early targets of QI efforts Improvement also seen in more diffuse domains E.g., Nurse Communication and Staff Responsiveness May reflect staffing changes 118

Observations on Least Improved Measures Less improvement seen in overall patient impressions (e.g. would Recommend the Hospital) Improving overall patient impressions may take more time May consider reputational factors Previous research suggests that performance on physician-related measures (e.g. Doctor Communication) can be difficult to change 119

Looking Ahead Improvements may continue and even accelerate in the future Incentives for improvement could become even stronger (proposed HVBP) Hospitals now have access to comparative information on HCAHPS performance Hospitals have had more time for QI projects HCAHPS scores are now used in various hospital rating and selection tools that are highly visible to consumers, such as Consumer Reports Increased visibility of HCAHPS scores may further increase selection into hospitals that report better patient experiences 120

Hospital Characteristics Associated with HCAHPS Scores [see Lehrman et al. Characteristics of Hospitals Demonstrating Superior Performance in Patient Experience and Clinical Process Measures of Care. Medical Care Research and Review 2010 Feb; 67(1):38-55. ] 121

Hospital Factors Examined Region (Census Division) Urban/Rural Location Bed size Ownership 122

Data Data from 2.2 million patients discharged July 2007-June 2008 from 3,844 hospitals in March 2009 public reporting 123

Analytic Approach Multi-level multiple regression was used to predict the 10 HCAHPS hospital-level measures from Hospital characteristics Individual patient-mix adjustor variables Prior to regression analysis, individual-level scores for each HCAHPS measure were adjusted for mode of survey administration 124

March 2009 Parameter Estimates for Responsiveness of Hospital Staff - Table 1 Beta Rural 1-99 beds 5.0* Rural 100-299 beds 3.0* Rural 200+ beds 1.7* Urban 1-99 beds 3.8* Urban 100-299 beds 1.1* [compared to Urban 200+ beds] (0) Government 2.8* Non-Profit other than Church operated 2.3* Church operated -0.3* [compared to For Profit] (0) *p<.001 125

March 2009 Parameter Estimates for Responsiveness of Hospital Staff - Table 2 Beta [compared to New England] (0) Mid Atlantic (NY/NJ/PA) -1.6* South Atlantic (VA to FL) -0.2 East North Central (e.g. IL,MI,OH) 0.1 East South Central (AL/MS/TN/KY) 1.8* West North Central ( Farm Belt ) 0.6* West South Central (TX/OK/AR/LA) 2.1* Mountain (Rocky Mtns and Southwest) -0.3 Pacific (CA/OR/WA/AK/HI) 0.1 *p<.001 126

HCAHPS Scores Varied by Bed Size & Ownership Bed size <100 was associated with significantly higher scores on all 10 HCAHPS measures (p<.001) for both relative to 200+ beds These effects particularly large for Staff Responsiveness, Cleanliness, and Quietness Clinical process measures show the opposite pattern Government hospitals had significantly higher scores on all 10 HCAHPS measures than For-Profit hospitals Other Non-Profit hospitals showed significantly higher scores for all HCAHPS measures except Quietness relative to For-Profit hospitals Clinical process measures show a similar pattern 127

HCAHPS Scores Varied by Region Lower HCAHPS scores were associated with hospitals in Mid-Atlantic (all 10 measures), Mountain (8 measures), and Pacific (8 measures) regions relative to reference category New England West South Central hospitals showed significantly higher scores than New England for 8 measures, largest for Quietness and Responsiveness of Staff Regional patterns are persistent and similar for Medicare Advantage (Plan) CAHPS Surveys 128

Racial/Ethnic Variation in HCAHPS Scores [see Goldstein et al. Racial/Ethnic Differences in Patients Perceptions of Inpatient Care Using the HCAHPS Survey. Medical Care Research and Review 2010 Feb; 67(1):74-92. ] 129

Objectives Compare inpatient experiences of Hispanic, African American, Asian/Pacific Islander, American Indian/Alaska Native, and multiracial inpatients to non-hispanic White inpatients Seek to understand role of between- and within-hospital differences in patient perspectives of hospital care 130

Data Data from 1.2million patients discharged October 2006-June 2007 from 2,684 hospitals in March 2008 public reporting 131

Racial/Ethnic Distribution 1% 1% 2% 3% 7% 7% Race/Ethnicity Distribution 79% non-hispanic White Hispanic African American American Indian/Alaska Native Asian/Pacific Islander Multi-racial Unknown 132

Patient Characteristics Overall N=1,203,229 Non-Hispanic White N=953,987 Hispanic N=83,283 African American N=85,564 American Indian N = 7,641 Male 37% 38% 28% 29% 37% 29% Age: 18-64 52% 48% 73% 67% 66% 66% Age: 65-74 20% 21% 13% 17% 19% 18% Age: 75+ 28% 31% 14% 16% 15% 16% Emergency Room Admission 38% 39% 30% 43% 38% 27% Maternity 13% 12% 31% 13% 32% 14% Surgical 35% 37% 28% 29% 32% 28% Medical 51% 52% 41% 58% 54% 40% Health: fair or poor 8 th grade or less More than 4 year college degree 28% 29% 22% 30% 34% 18% 7% 5% 18% 8% 9% 7% 12% 12% 7% 8% 7% 24% Asian/Pacific Islander N=22,106 133

Methods Used 8 HCAHPS Measures (all but Rating and Recommendation) First model predicted HCAHPS measures from race/ethnicity, controlling for survey mode and patient mix Estimated overall racial/ethnic differences HCAHPS patient-mix-adjustment does not adjust for race/ethnicity Second model added hospital fixed effects to measure racial/ethnic differences within the same hospitals 134

Hispanics: Total Differences from Whites Hispanics reported generally less positive experiences than non-hispanic Whites Nurse Communication: -0.77 points Staff Responsiveness: -1.47 Discharge Information: -2.12 Cleanliness: -1.36 Quietness: +.84 (p<0.001 for all differences noted) 135

Hispanics: Within-Hospital Differences However, within the same hospitals, Hispanics consistently reported more positive experiences than non-hispanic Whites Nurse Communication: +1.39 points Doctor Communication: +1.41 Staff Responsiveness: +1.42 Pain Management: +1.80 Communication about New Medicines: +2.70 Quietness: +3.18 (p<0.001 for all differences noted) 136

African Americans: Total Differences from Whites Findings for African Americans were mixed compared to non-hispanic Whites Nurse Communication: +0.85 points Doctor Communication: +1.43 Staff Responsiveness: -1.17 Pain Management: +1.10 Communication about Medicines: +1.84 Discharge Information: -1.84 Cleanliness: -0.53 Quietness: +7.68 (p<0.001 for all differences noted) 137

African Americans: Within-Hospital Differences African Americans reported consistently more positive experiences than non-hispanic Whites within the same hospitals Nurse Communication: +2.60 points Doctor Communication: +2.12 Staff Responsiveness: +1.50 Pain Management: +2.48 Communication about Medicines: +4.19 Cleanliness: +1.73 Quietness: +8.15 (p<0.001 for all differences noted) 138

Asians/Pacific Islanders: Total Differences from Whites Asians reported consistently less positive experiences than non-hispanic Whites Nurse Communication: -2.57 points Doctor Communication: -1.77 Staff Responsiveness: -4.06 Pain Management: -3.48 Discharge Information: -1.94 Cleanliness: -2.40 Quietness: -0.75 (p<0.001 for all differences noted) 139

Asians/Pacific Islanders: Within-Hospital Differences Within-hospital findings for Asians were mixed compared to non-hispanic Whites Nurse Communication: -0.59 Staff Responsiveness: -1.71 Pain Management: -2.04 Communication about Medicines: +1.38 Cleanliness: -1.09 Quietness: +2.71 (p<0.001 for all differences noted) 140

American Indians: Total Differences from Whites Findings for American Indians/Alaska Natives were mixed compared to non-hispanic Whites Nurse Communication: -1.17 points Doctor Communication: -1.00 Communication about Medicines: +2.20 Quietness: +2.64 (p<0.001 for all differences noted) 141

American Indians: Within-Hospital Differences Within-hospital differences were similar to the overall differences for American Indians/Alaska Natives Nurse Communication: -0.93 points Doctor Communication: -1.00 Communication about Medicines: +2.00 Quietness: +2.31 (p<0.001 for all differences noted) 142

Summary of Racial/Ethnic Variation in HCAHPS Scores Overall, Hispanics and Asians reported generally less positive experiences than non-hispanic Whites Non-Hispanic Whites received care at hospitals that delivered better average patient experiences to all patients, compared to hospitals at which African American, Hispanic, and Asian patients received care Within the same hospitals Patients reported generally similar experiences with African Americans and Hispanics often reporting better experiences than non-hispanic Whites Asians generally reported poorer experiences than non-hispanic Whites 143

Questions? 144

HCAHPS and Proposed Hospital Value-Based Purchasing (HVBP) 145

Hospital Value-Based Purchasing Program (HVBP) Required by the Affordable Care Act Applies to Medicare payments under the Inpatient Prospective Payment System (IPPS) for inpatient stays Dramatically changes how Medicare pays hospitals Proposed rule published on January 7, 2011 Comment period closes on March 8, 2011 146

Timing Begins in FY 2013 for acute care hospitals Applies to payment for discharges occurring on or after October 1, 2012 147

Based on: Incentive Payments How well hospitals perform on certain quality measures (achievement) and How much the hospitals performance improves (improvement) 148

Proposed Performance Period Baseline Period for FY 2013: July 1, 2009 through March 31, 2010 Performance Period for FY 2013: July 1, 2011 through March 31, 2012 Performance Period for future years: based on 4 quarters of data 149

Initial Measures Proposed to be Included HCAHPS Clinical Measures Acute Myocardial Infarction Heart Failure Pneumonia Surgical Care Improvement Project 150

HCAHPS Scoring Proposing that a hospital needs to report at least 100 HCAHPS completed surveys to be included in Hospital VBP 151

Eight Proposed HCAHPS Dimensions Nurse Communication Doctor Communication Cleanliness and Quietness Responsiveness of Hospital Staff Pain Management Communication about Medications Discharge Information Overall Rating 152

Proposed HCAHPS Scoring Methodology Achievement points Improvement points Consistency points 153

Achievement Thresholds Achievement threshold is proposed to be set at the 50 th percentile of hospital performance on the dimension during the baseline period Hospitals will have to score at or above this threshold to earn achievement points Benchmark corresponds to excellent performance in the baseline period A hospital will be awarded maximum achievement points if the hospital performed at least at the 95 th percentile of performance during the baseline period 154

Achievement Points Propose to assign 0 to 10 points for each of the 8 HCAHPS dimensions as follows: Hospital s score on a dimension is = or > than benchmark, hospital receives 10 points Hospital s score is within achievement range, the hospital would receive a score of 1 to 9 using the following formula: ((Hospital HCAHPS performance period dimension score (percentile) 50)/5) + 0.5 Hospital s score is less than the achievement threshold (50 th percentile), hospital receives 0 points 155

Improvement Points A hospital can earn from 0 to 9 improvement points for each HCAHPS dimension as follows: If a hospital s score is greater than its baseline period score but below the benchmark, the hospital would receive a score of 0 to 9 using the following formula: [10*((Hospital performance period score Hospital baseline period score)/(benchmark Hospital baseline period score))] 0.5 If a hospital s score is lower than its baseline period score, there is no improvement, or if the baseline period score was already at the benchmark, the improvement score is 0 156

HVBP: Consistency Points Consistency score recognizes consistent achievement across dimensions Propose a maximum of 20 consistency points that would be awarded proportionately based on the lowest of a hospital s 8 HCAHPS dimension scores during the performance period compared to the median baseline period score for each HCAHPS dimension assigned according to the following formula: (2*(lowest percentile/5)) 0.5 157

Overall HCAHPS Performance Score For each of the 8 dimensions, determine the larger of the 0 to 10 achievement score and the 0 to 9 improvement score to create 8 dimension scores Sum the 8 dimension scores to arrive at a 0 to 80 HCAHPS base score Calculate the 0 to 20 consistency score Total overall score = base score + consistency score 158

HVBP: Total Performance Score HCAHPS performance score proposed to account for 30% of the total Hospital VBP performance score 159

Questions? 160

Contact Us HCAHPS Information and Technical Support Web site: E-mail: www.hcahpsonline.org hcahps@azqio.sdps.org Telephone: 1-888-884-4007 161