As organizations strive to improve

Specil Article Refinement, scoring, nd vlidtion of the Fmily Stisfction in the Intensive Cre Unit (FS-ICU) survey* Richrd J. Wll, MD, MPH; Ruth A. Engelberg, PhD; Lois Downey, MA; Dren K. Heylnd, MD, MSc; J. Rndll Curtis, MD, MPH Objectives: To refine the Fmily Stisfction in the Intensive Cre Unit (FS-ICU) survey nd develop vlidted method for scoring the instrument. Design: Instrument development study, using dt from two prospective cohort studies. Setting: Intensive cre units in seven university-ffilited hospitls (six Cndin, one United Sttes). Subjects: Fmily members of ICU ptients. Interventions: Bsed on priori criteri, items were tgged for potentil removl nd discussed with the FS-ICU developers. Fctor nlysis ws used to test the conceptul structure of the instrument nd develop scoring method bsed on scles nd subscles. The new scoring method ws vlidted in the U.S. cohort using the Qulity of Dying nd Deth (QODD) instrument nd nurse-ssessed qulity indictors. Mesurements nd Min Results: A totl of 1,038 fmily members completed the FS-ICU cross seven sites. Fifteen items were initilly tgged for possible removl. After consensus with the developers, ten items were dropped (nd 24 were retined in the finl instrument). Fctor nlysis explined 61.3% of the totl vrince using two-fctor model. The first fctor pertined to stisfction with cre (14 items). The second fctor encompssed stisfction with decision mking (10 items). A scoring method ws developed bsed on this conceptul model. In vlidity testing, the FS-ICU ws significntly correlted with the Fmily-QODD totl score (Spermn s.56, p <.001) s well s individul QODD items such s qulity of cre by ll providers (.64, p <.001). The FS-ICU lso correlted significntly with multiple nurse-ssessed qulity indictors. Conclusions: The shortened FS-ICU mesures two min conceptul domins stisfction with cre nd stisfction with decision mking. Scores on the FS-ICU show good vlidity ginst other indictors of ICU qulity. The instrument holds promise s useful outcome mesure in studies tht ttempt to improve this component of ICU cre. (Crit Cre Med 2007; 35:271 279) KEY WORDS: criticl cre; fmily reserch; fmily stisfction; consumer stisfction; relibility nd vlidity *See lso p. 324. From Hrborview Medicl Center, Division of Pulmonry nd Criticl Cre, Deprtment of Medicine, University of Wshington, Settle, WA (RJW, RAE, LD, JRC); nd Deprtment of Medicine, Kingston Generl Hospitl, Queen s University, Kingston, Ontrio, Cnd (DKH). Supported, in prt, by grnt from the Ntionl Institute of Nursing Reserch (R01NR05226), Bethesd, MD. The uthors hve not disclosed ny potentil conflicts of interest. Copyright 2006 by the Society of Criticl Cre Medicine nd Lippincott Willims & Wilkins DOI: 10.1097/01.CCM.0000251122.15053.50 As orgniztions strive to improve the qulity of cre in the intensive cre unit (ICU), they require tools tht mesure the vrious dimensions of qulity. According to the Institute of Medicine, qulity cre is sfe, timely, efficient, effective, equitble, nd ptient-centered (1). Improving qulity of cre requires tools tht ccurtely mesure these ttributes. In the ICU, ptient-centered cre includes fmily-centered cre (2). Since most ICU ptients cnnot mke decisions for themselves (3, 4), fmilies re often involved s surrogte decision mkers. Therefore, the perspectives of fmily nd other surrogte decision mkers re especilly importnt in the criticl cre setting, nd fmily stisfction is n importnt outcome mesure. The Fmily Stisfction in the ICU survey (FS-ICU) ws developed nd vlidted for ssessing fmily stisfction with cre in the ICU (5). The instrument hs been successfully dministered in multiple-center study cross Cnd, suggesting good potentil for widespred use (6). Others hve lso successfully used the FS-ICU to led qulity improvement inititives nd study surrogte decision mking in the ICU (7, 8). More recently, modified FS-ICU ws used by the Americn College of Chest Physicins in multiple-center intervention study (9 11). The instrument hs lso been trnslted into Spnish, Germn, nd French (12). Overll, the FS-ICU is comprehensively developed, widely vilble, nd well-tested tool for mesuring fmily stisfction in the ICU. In ddition to the FS-ICU, two other vlidted instruments re vilble for mesuring the qulity of cre delivered to fmilies in the ICU. The best known tool for ssessing fmily needs in the ICU is the 14-item Criticl Cre Fmily Needs Inventory (13, 14). Although this importnt instrument hs been rigorously evluted during the pst 20 yrs, meeting fmily needs does not necessrily gurntee high fmily stisfction (6). For this reson, developing tool to specificlly mesure fmily stisfction is n importnt undertking for both reserch nd ICU qulity improvement. The Criticl Cre Fmily Stisfction Survey is 20-item questionnire designed to mesure fmily stisfction in the ICU (15). Although recent study showed tht one domin of this instrument correltes well with fmily rtings of end-of-life cre (16), reserch experience with this tool is still limited. Despite the populrity of the FS-ICU, systemtic method for computing totl Crit Cre Med 2007 Vol. 35, No. 1 271

FS-ICU score hs not yet been undertken. In ddition, the instrument s psychometric properties hve not been rigorously evluted in non-cndin ICU popultion. The purpose of this rticle ws two-fold: ) to develop scoring method for the FS-ICU bsed on scles nd subscles; nd b) to vlidte the instrument nd the new scoring method using non-cndin smple. These steps, if successful, could estblish the FS-ICU s potentilly importnt outcome mesure for rndomized trils nd qulity improvement efforts. MATERIALS AND METHODS The Originl FS-ICU The FS-ICU ws developed to mesure fmily stisfction with cre provided in the ICU. A detiled description of the questionnire s development, relibility testing, nd vlidtion hs been published elsewhere (5). Briefly, the originl FS-ICU consists of 34 items generted from conceptul frmeworks of ptient stisfction, qulity end-of-life cre, reserch on needs of criticlly ill fmilies, literture on fmily stisfction with medicl decision mking, existing vlidted stisfction surveys, nd pilot study. The questionnire ws designed with two conceptul sections the first prt focuses on stisfction with overll cre (18 items) nd the second prt ssesses stisfction with decision mking (16 items). During development, items were pretested for clrity, redbility, nd content vlidity with clinicins nd fmily members of criticlly ill ptients t Cndin cdemic hospitl. Internl vlidity ws ssessed with 22 fmily members of mechniclly ventilted ptients who hd been in the ICU 48 hrs. Cronbch s lph (internl consistency) for the questionnire s sections rnged from.74 to.95, nd test-retest relibility with 25 fmily members t 7 10 dys ws 0.85 (5). Appendix 1 displys the individul FS-ICU items, nd the full instrument is vilble online (12). Popultions nd Settings Our current study used combined (Cndin/U.S.) popultion tken from two prior independent prospective studies. Both studies were pproved by their respective hospitls Institutionl Review Bords. Cndin Cohort. A prospective study to determine the level of stisfction mong fmilies of criticlly ill ptients in six tertiry Cndin hospitls hs been previously described (6). Prticipting ICUs vried in size from eight to 24 beds. The eligible study smple included 891 consecutive consenting fmily members (next of kin or decision mkers) of mechniclly ventilted dult ptients. The person who completed the FS-ICU hd visited the ptient t lest once during the ICU sty. Fmily members were pproched t the time of dischrge from the ICU (for survivors) or by mil 4 wks fter deth (for nonsurvivors). The miled pcket included cover letter tht expressed sympthy nd explined the study, the originl FS-ICU, nd stmped return envelope. Four weeks lter, second questionnire ws sent to those who hd not responded. The response rte for this study ws 70% (n 624). Following study closure (nd publiction), n dditionl 191 fmily members completed the FS-ICU. Our nlyses use this totl smple (n 815). U.S. Cohort. A prospective study to evlute the qulity of pllitive cre for ICU ptients nd their fmilies ws conducted t Hrborview Medicl Center, 350-bed tertiry hospitl ffilited with the University of Wshington. The hospitl hs six distinct ICUs (medicl, trum, neurosurgicl, surgicl, coronry, nd combined burn-peditric). We prospectively identified ptients who hd been in the ICU for 6 hrs nd dministered the FS-ICU to their consenting fmily members (next of kin or decision mkers). If the ptient died in the ICU (or within 24 hrs of ICU trnsfer), we miled pcket to the fmily member 4 wks fter the ptient s deth. The miled pcket included cover letter expressing sympthy nd explining the study, the originl FS-ICU, the Qulity of Dying nd Deth (QODD) questionnire (17 21), nd stmped return envelope. A postcrd reminder/thnk-you ws sent 2 wks fter the originl miling nd, if questionnire ws still not returned, second questionnire (with cover letter) ws sent 3 wks lter. If the ptient survived 24 hrs fter ICU dischrge, we gve the FS-ICU directly to the fmily member while the ptient ws still in the hospitl or, if we did not mke contct with the fmily, we miled pcket to the fmily fter dischrge from the hospitl using the sme follow-up s described for fmily of ptients who died. We lso identified the nurse cring for the ptient t the time of ICU dischrge or deth nd dministered surveys sking bout nursing indictors of ICU qulity (described subsequently). For nonsurvivors, we lso surveyed the nurse cring for the ptient during the shift before deth, in cse the nurse t the time of deth did not hve sufficient time to evlute the ptient s experiences. Nurse surveys were distributed within 72 hrs of ICU dischrge or deth. The number of fmily members recruited from ech of the vrious ICUs rnged from 47 to 125, nd the response rte ws 223 of 510 (44%). The response rte for nurse surveys ws 469 of 746 (63%). Dt Anlysis, Objective 1 Objective 1 ws to develop scoring method for the FS-ICU bsed on scles nd subscles. For this objective, five nlytic steps were performed on the combined popultion: ) descriptive nlysis; b) item reduction; c) fctor nlysis; d) relibility nlysis; nd e) developing scoring lgorithm. We used n itertive pproch: Items were deleted bsed on findings from the descriptive, fctor, nd relibility nlyses nd we then repeted ll five steps with the reduced set of items. Descriptive Anlyses. Item descriptive nlyses were completed t individul sites nd then cross ll sites combined. These included frequencies, percent ceiling/floor scores, percent missing responses, medins, nd interqurtile rnges (IQRs). Item Reduction. Vrious pproches hve been outlined for shortening questionnires (22). One of our gols ws to crete shorter instrument tht cn be more esily completed by fmilies of ICU ptients. However, we lso wnted to preserve the mesurement properties of the originl FS-ICU. Bsed on priori criteri, we identified items tht chllenged scle integrity nd tgged them for possible removl. Our criteri included the following: 1. Globl items tht were designed for vlidity testing during instrument development 2. Items with high nonresponse rtes, defined s 10% missing 3. Items with less discrimintion, defined s 70% endorsement of the lowest (floor) or highest (ceiling) possible score 4. Redundnt items, s suggested by item-scle Cronbch s.8 (corrected for overlp) 5. Items mesuring nother construct, s determined by lodings 0.4 in principl component nlysis After tgging removble items, we emiled the originl FS-ICU developers with detiled explntion of our findings nd other sttisticl results (described subsequently). Severl weeks lter, we discussed these issues vi conference cll with the FS-ICU developers nd estblished consensus on finl shortened version of the instrument. Fctor Anlysis. Principl component nlysis ws used to identify items for possible removl (23). Then we performed explortory fctor nlysis for ech site seprtely, using identicl procedures for dt imputtion, fctor extrction, nd stndrd vrimx orthogonl rottion. Since fctor nlysis cnnot be performed with missing dt, we used mrginl medin substitution for FS-ICU items with 15% nonresponse. To minimize the bis on our vrince estimtes, items with 15% missing were not imputed. Becuse fctor nlysis cn be bised when vribles re on n intervl scle, we bsed our nlyses on polychoric correltions, which ssume tht intervl level dt re representtive of n underlying, continuous distribution (24). Before we performed fctor nlysis, five items with discontinuous response options were recoded 272 Crit Cre Med 2007 Vol. 35, No. 1

into ordinl scles nd were then linerly trnsformed to hve the sme response vlues s the rest of the FS-ICU items. Three items were recoded s dichotomous vribles (numbers 26, 27, nd 32) nd two items were recoded to Likert-3 scle (numbers 25 nd 30). A detiled explntion of item recoding is vilble on the FS-ICU Website (12). We decided priori to extrct the number of fctors determined by 1,000 rndom dt prllel nlyses (25). Cttell s scree test ws secondrily checked for greement (26). Both pproches indicted tht two-fctor model ws pproprite. After running nlyses seprtely on ech site, we tested for similrity between individul fctor structures using the coefficient of congruence (27). This confirmed we could combine sites, nd the fctor nlytic procedures were rerun with ll sites combined (n 1,038 fmily members). Following item reduction, we repeted these nlyses on the finl shortened FS-ICU. The principl component nd fctor nlyses were performed using MicroFACT 2.1 (Assessment Systems Corportion, St. Pul, MN). Relibility Anlysis. Bsed on the new fctoril structure, we reevluted internl consistency in the combined U.S./Cndin popultion, clculting item-subscle nd itemtotl s (28). To ssess if the subscles could be combined into totl score, we checked univrite correltion between the two scles using the more conservtive, nonprmetric Spermn s correltion coefficient. We did not repet test-retest relibility. After item reduction, we repeted ll of these relibility nlyses on the finl set of items. Scoring. Items were oriented so tht higher vlues indicted incresed stisfction. We performed liner trnsformtions to mke item vlues more meningful nd to stndrdize the response scle cross ll items: trnsformed vlue ([ctul item vlue lowest possible item vlue]/[possible item rnge]) 100 (29). Thus, the trnsformed vlues rnged from 0 to 100. We clculted subscle nd totl scores by verging vilble items, provided the respondent nswered 70% of the items in the respective subscle/totl. Scores were expressed s medin nd IQR to ccount for dt skewness. Dt Anlysis, Objective 2 Objective 2 ws to vlidte the scoring method. Conceptul Approch to Vlidity Anlyses. Vlidity is mesure of the degree of confidence one cn plce in the inferences drwn from scores on n instrument (30). An underlying ssumption is tht delivering highqulity cre improves fmily stisfction in the ICU. We hypothesized tht higher FS-ICU scores would correlte with higher scores on nursing nd fmily indictors of ICU qulity (selected priori). These hypotheses were tested in the U.S. cohort becuse this ws the Tble 1. Chrcteristics of ll ptients nd fmily respondents (n 1,038) cohort tht completed the construct vlidtion instruments. To determine nursing indictors of ICU qulity, nurses were sked to complete the previously vlidted nurse-ssessed QODD (17 21). Nurses were lso sked to complete three questionnires ssessing fmily-focused cre in the ICU. These included 17-item questionnire ssessing nursing ctivities performed for ICU fmilies, 14-item questionnire ssessing nurse perceived brriers to delivering cre to fmily members, nd fouritem questionnire rting nurse stisfction with meeting fmily needs. The full instruments re vilble online (31). Items in these instruments were generted through review of the literture nd ides provided by focus groups with 21 criticl cre nurses (13, 14, 32). A description of the development of the first two questionnires hs been published (33), nd ll three questionnires provide totl score using n pproch tht ws recently vlidted with principl component nlyses (34). When two nurses returned questionnires for the sme ptient, we verged scores between the respondents. We hypothesized tht higher scores on the QODD, ctivities, nd meeting needs questionnires would be ssocited with higher scores on the FS-ICU. We hypothesized tht incresed brriers to delivering cre would be ssocited with lower scores on the FS-ICU. To determine fmily indictors of ICU qulity, fmily members of nonsurvivors lso completed the QODD. We used this instrument becuse it is vlidted indictor of ICU qulity with end-of-life cre (17 21). We hypothesized tht higher scores on the QODD would be significntly ssocited with higher scores on the FS-ICU. In ddition to the totl QODD score, we lso exmined four QODD items tht we hypothesized would correlte significntly with the FS-ICU. Those items were pin control, brething comfort, cre by doctors, nd cre by ll providers. We expected tht the FS-ICU would hve stronger ssocition with the Fmily-QODD thn with the Nurse-QODD, given the different perspectives nd experiences of fmily members nd nurses. We lso hypothesized tht QODD items would correlte more strongly with the Stisfction with Cre subscle thn with the Stisfction With Decision Mking subscle. Sttisticl Anlysis for Vlidtion. Becuse the distributions of questionnire scores were nonnormlly distributed, we used the nonprmetric Spermn correltion coefficients to test our vlidtion hypotheses. Acknowledging the multiple comprisons in these explor- Chrcteristics Ptients Fmily Age, yrs, men (SD) 61 (17.9) 54.2 (14.4) Femle, no. (%) 414 (39.9) 631 (60.8) Rce/ethnicity, no. (%) White 922 (88.9) Asin/Pcific Islnder 39 (3.8) Ntive Americn/Alskn 31 (3) Blck/Africn Americn 8 (0.8) Hispnic 1 (0.1) Other/unknown 37 (3.6) Primry dignosis, no. (%) Crdiovsculr 188 (18.1) Respirtory 280 (27) Neurologic 133 (12.8) Sepsis 82 (7.9) Trum 132 (12.7) Gstroenterology 131 (12.6) Other 92 (8.9) APACHE II, men (SD) 22.9 (8) ICU LOS, medin dys (rnge) 8.3 (1 138) Reltionship to ptient, no. (%) Spouse or prtner 481 (46.3) Child 279 (26.9) Prent 108 (10.4) Sibling 91 (8.8) Other 79 (7.6) Lives with ptient, no. (%) 602 (58) Level of eduction, no. (%) 8th grde or less 126 (12.1) High school 340 (32.8) Post secondry (college or grdute) 511 (49.2) Other/unknown 61 (5.9) APACHE, Acute Physiology nd Chronic Helth Evlution; ICU, intensive cre unit; LOS, length of sty. Only vilble for Cndin cohort (n 815). Totls my exceed 100% due to rounding. Crit Cre Med 2007 Vol. 35, No. 1 273

Tble 2. Descriptive sttistics nd internl consistency for individul FS-ICU items (n 1,038 respondents) Item % Missing % Floor % Ceiling Medin Score IQR Men Score SD Corrected Item- Subscle Cronbch s 1 Courtesy, respect, nd 0.4 0.6 70.2 100 75 100 90.9 16.2.73 compssion towrd ptient. 2 Mngement of pin. 3.9 0.6 60.5 100 75 100 88.0 18.0.68 3 Mngement of brethlessness. 10.1 0.9 58.0 100 75 100 87.9 19.3.64 4 Mngement of gittion. 10.8 1.5 50.3 100 75 100 84.0 22.3.68 5 How well stff considered fmily 1.3 1.9 55.5 100 75 100 83.9 22.6.76 needs. 6 How well stff provided emotionl 2.9 2.1 49.3 100 75 100 81.0 24.0.74 support towrd fmily. 7 How well stff met fmily 28.3 2.2 33.8 75 75 100 77.7 26.5 spiritul/religious needs. 8 Coordintion nd temwork by 1.3 0.3 60.3 100 75 100 86.7 19.6.77 stff. 9 Courtesy, respect, nd 0.9 1.5 58.8 100 75 100 85.3 21.7.79 compssion towrd fmily. 10 Skill nd competence of nurses. 0.8 0.2 71.7 100 75 100 91.5 15.3.68 11 Communiction by nurses. 1.0 1.2 58.4 100 75 100 84.9 22.0.67 12 Skill nd competence of doctors. 1.8 0.8 60.3 100 75 100 86.6 20.1.62 13 Frequency of communiction by 2.0 5.1 36.2 75 50 100 70.5 29.6.66 doctors. 14 How well socil workers ssisted 36.5 4.3 26 75 50 100 72.8 30.1 nd supported fmily. 15 How well chplin ssisted nd 40.4 2.8 28.3 75 75 100 77.4 27.5 supported fmily. 16 Cre: tmosphere of the ICU. 1.7 1.2 43.6 75 75 100 79.4 22.6.71 17 Cre: tmosphere of the ICU 4.6 7.6 24 75 50 100 62.2 30.7.43 witing room. 18 Overll stisfction with ICU 0.4 1.3 52.4 100 75 100 83.5 21.3 experience. 19 Willingness of stff to nswer 1.1 1.3 54.5 100 75 100 84.2 21.4.69 questions. 20 Stff provided understndble 1.2 0.5 50 100 75 100 82.9 20.7.70 explntions. 21 Honesty of informtion provided 1.2 2.1 54 100 75 100 83.2 23.0.72 bout ptient s condition. 22 Completeness of informtion 1.3 1.9 51.6 100 75 100 81.9 23.8.76 bout wht ws hppening. 23 Consistency of informtion bout 3.1 3.6 42.1 75 75 100 76.9 26.3.74 ptient s condition. 24 Feel included in the decisionmking 2.4 7.4 49.4 100 50 100 75.8 31.1.48 process. 25 Involved t right time in 8.7 3.7 79 100 100 100 91.2 23.7 decision-mking process. 26 Received pproprite mount of 7.3 11 81.7 100 100 100 88.1 32.3 informtion. 27 Hd enough time to think in 9.3 10.6 80.1 100 100 100 88.3 32.1 decision-mking process. 28 Feel supported during the 7.1 4.9 26.8 75 50 100 71.2 27.2.52 decision-mking process. 29 Feel control over the cre of the 5.0 6.7 31.7 75 50 100 67.6 30.5.56 ptient. 30 Given right mount of hope 6.6 9.2 70.5 100 100 100 82.9 32.5 ptient would recover. 31 Agreement within fmily 4.9 1.1 41.5 75 75 100 79.1 23.5 regrding cre ptient received. 32 Adequte time to ddress 8.1 10.4 81.5 100 100 100 88.7 31.7.38 concerns & nswer questions. 33 Stisfction with level or mount 1.9 2.7 49.8 100 75 100 82.0 23.3.50 of cre ptient received. 34 Overll stisfction with decisionmking. 4.7 4.3 38.1 75 50 100 75.0 27.0 IQR, interqurtile rnge; ICU, intensive cre unit. Dropped from survey during item reduction. 274 Crit Cre Med 2007 Vol. 35, No. 1

tory nlyses, we defined (two-tiled) p.01 s sttisticlly significnt. Anlyses were performed using SPSS 13.0 (Chicgo, IL). RESULTS Objective 1: Scoring the FS-ICU Descriptive Anlyses. A totl of 1,038 fmily members t seven medicl centers (six Cndin, 1 United Sttes) completed the FS-ICU (Tble 1). The number of respondents t the individul sites rnged from 80 to 223. Tble 2 presents the descriptive sttistics for the 34 items from ll sites. Item Reduction. Fifteen items were tgged for possible removl nd discussed with the originl FS-ICU uthors. These included five items tht hd missing responses for 10%: dyspne mngement (number 3), gittion mngement (number 4), spiritul support (number 7), socil work (number 14), nd pstorl cre (number 15). We dropped three (numbers 7, 14, nd 15) nd retined two of these items in the finl instrument. We lso reviewed seven items with ceiling scores endorsed t 70%: compssion nd respect shown to the ptient (number 1), stisfction with nursing cre (number 10), receiving the right mount of informtion in the decision-mking process (number 26), hving enough time to think bout informtion provided (number 27), being involved t the right time in the decision-mking process (number 25), being given the right mount of hope for recovery (number 30), nd hving enough time to ddress concerns nd nswer questions (number 32). We dropped four (numbers 25, 26, 27, nd 30) nd retined three of these items in the finl instrument. A single item ssessing greement within the fmily (number 31) loded wekly in principl component nlysis (lod 0.32) nd ws dropped. Lst, we dropped two globl rting items since they hd been included for vlidity testing during instrument development nd were redundnt (numbers 18 nd 34). The remining 24 items were retined for fctor nlysis. The detiled rtionle for ech item s retention or removl is vilble online (12). Fctor Anlysis. The finl fctor nlysis model reveled tht 14 of 24 items loded on first fctor pertining to stisfction with cre of the ptient nd fmily nd the remining 10 items loded onto second fctor tht encompssed stisfction with decision mking (Tble Crit Cre Med 2007 Vol. 35, No. 1 Tble 3. Fctor nlysis using two-fctor model Item Fctor 1 Fctor 2 1 0.816 0.288 Cre: courtesy, respect, nd compssion by stff towrd ptient. 2 0.816 0.166 Cre: mngement of pin. 3 0.773 0.212 Cre: mngement of brethlessness. 4 0.780 0.218 Cre: mngement of gittion. 5 0.707 0.455 Cre: how well stff considered fmily needs. 6 0.682 0.445 Cre: how well stff provided emotionl support towrd fmily. 8 0.799 0.347 Cre: coordintion nd temwork by stff. 9 0.749 0.438 Cre: courtesy, respect, nd compssion by stff towrd fmily. 10 0.780 0.279 Cre: skill nd competence of nurses. 11 0.651 0.458 Cre: communiction by nurses. 12 0.618 0.433 Cre: skill nd competence of doctors. 16 0.692 0.373 Cre: tmosphere of the ICU. 17 0.418 0.323 Cre: tmosphere of the ICU witing room. 33 0.521 0.407 Cre: stisfction with level or mount of cre ptient received. 13 0.438 0.628 DM: frequency of communiction by doctors. 19 0.186 0.570 DM: willingness of stff to nswer questions. 20 0.585 0.587 DM: stff provided understndble explntions. 21 0.564 0.619 DM: honesty of informtion provided bout ptient s condition. 22 0.627 0.624 DM: completeness of informtion bout wht ws hppening. 23 0.538 0.653 DM: consistency of informtion bout ptient s condition. 24 0.122 0.641 DM: feel included in the decision-mking process. 28 0.226 0.623 DM: feel supported during the decision-mking process. 29 0.199 0.651 DM: feel control over the cre of the ptient. 32 0.186 0.570 DM: dequte time to ddress concerns nd nswer questions. Cre, stisfction with cre (first fctor); ICU, intensive cre unit; DM, stisfction with decision mking (second fctor). Boldfce, fourteen items lod on the first fctor (Stisfction with Cre), nd ten items lod on the second fctor (Stisfction with Decision Mking). 3). Together, these two fctors explined 61.3% of the observed vrince. We tested the robustness of our findings by performing three-fctor model, nd this third fctor only explined n dditionl 3.4% of the vrince. Four items ssessing informtion exchnge (numbers 20 23) loded eqully on both fctors. Although the instrument s developers hd envisioned informtion exchnge s prt of decision mking (35), we felt tht our finding mde good conceptul sense becuse the degree to which fmily s informtion needs re met in the ICU will lso ffect fmily s stisfction with cre. When developing our subscles, however, we elected to uphold the originl conceptul frmework outlined by the instrument s developers for these items nd we grouped them in the decision mking subscle. Relibility Anlysis. The Cronbch s coefficients were.92 nd.88 for the Stisfction with Cre nd the Stisfction With Decision Mking subscles, respectively. The two subscles showed good correltion with ech other (Spermn s 0.73, p.001), suggesting tht single scle for the entire instrument ws resonble. Cronbch s coefficient for this single scle ws.94. Itemsubscle correltions (corrected for overlp) rnged from.38 to.79 (Tble 2). No items met redundncy criteri for removl (.8). Scoring. Bsed on the two-fctor model, we developed two subscle scores (FS-ICU/Cre nd FS-ICU/DM) nd totl instrument score (FS-ICU/Totl). The mjority of respondents were stisfied with their overll ICU experience, nd the medin FS-ICU/Totl score (IQR) ws 85.4 (72.9 93.8). The medin scores (IQR) for the FS-ICU/Cre nd FS- ICU/DM subscles were 88.5 (75 96.4) nd 82.5 (70 92.5), respectively. 275

Tble 4. Chrcteristics of ptients nd respondents in the U.S. cohort Chrcteristics Objective 2: Vlidtion Tble 4 presents self-reported respondent nd chrt-bstrcted ptient chrcteristics for the U.S. cohort. There were no significnt ge or gender differences mong ptients of respondents nd nonrespondents. Ptients of respondents hd slightly longer men ( SD) ICU lengths of sty (8 10 vs. 6 9 dys) nd were more likely to be white (75% vs. 66%, p.03). Among 223 fmily members completing the FS-ICU, 125 fmily members lso completed the Fmily-QODD becuse their loved one died in the ICU. Overll, the FS-ICU showed moderte to strong correltion with the totl Fmily-QODD score s well s the individul items selected priori (Tble 5). As hypothesized, Ptients (n 223) No. (%) ll individul QODD items correlted more hevily with the Cre subscle thn the Decision Mking subscle. The FS-ICU correlted significntly with severl nurse rtings of the ICU qulity, including ssessments of meeting fmily needs nd perceived brriers to cre. The FS-ICU did not correlte significntly with the Nurse-QODD or the totl number of nursing ctivities performed for fmilies. As hypothesized, the FS-ICU correlted more highly with the Fmily- QODD thn with nurse ssessments of the ICU experience. DISCUSSION Fmily (n 223) No. (%) Nurses (n 209) No. (%) Age, yrs, men (SD) 56 (21.4) 53.1 (13.3) 39 (8.2) Femle, n (%) 82 (36.8) 145 (65) 170 (81.3) Rce/ethnicity, n (%) White 168 (75.3) 175 (78.5) 182 (87.1) Asin/Pcific Islnder 13 (5.8) 21 (9.4) 18 (8.6) Ntive Americn/Alskn 10 (4.5) 13 (5.8) 3 (1.4) Blck/Africn Americn 4 (1.8) 14 (6.3) 2 (1) Hispnic 1 (0.4) 8 (3.6) 2 (1) Other/Unknown 27 (12.2) 6 (2.7) 10 (4.8) Primry dignosis, n (%) Crdiovsculr 23 (10.3) Respirtory 16 (7.2) Neurologic 60 (26.9) Sepsis 6 (2.7) Trum 71 (31.8) Gstroenterology 19 (8.5) Other 28 (12.6) ICU LOS, medin dys (rnge) 4 (1 64) Hospitl LOS, medin dys (rnge) 7 (1 113) Reltionship to ptient, n (%) Spouse or prtner 86 (38.6) Child 56 (25.1) Prent 41 (18.4) Sibling 18 (8.1) Other 22 (9.9) Yers known ptient, men (SD) 35 (17.4) Lives with ptient, n (%) 124 (55.6) Level of eduction, n (%) 8th grde or less 12 (5.4) High school 44 (19.7) Post secondry (college or grdute) 163 (73.1) Other/unknown 4 (1.8) Nurse level of trining, n (%) Diplom 9 (4.3) Associte degree 48 (23) Bcclurete 141 (67.5) Mster s 8 (3.8) Other 3 (1.4) Yers of nursing, men (rnge) 12.3 (1 34) Yers of criticl cre nursing, men (rnge) 8.9 (1 29) ICU, intensive cre unit; LOS, length of sty. Sums my exceed 100% becuse individuls identify two rcil/ethnic bckgrounds. Fmily stisfction is n importnt mesure of the qulity of ICU cre (2, 36 38). In this popultion involving 1,038 fmilies from seven medicl centers, we describe psychometric properties of the FS-ICU, n instrument designed to mesure fmily stisfction in the ICU. We used fctor nlysis to develop n empiriclly supported scoring method for the FS-ICU bsed on totl score nd two subscle scores. We then vlidted the FS-ICU s n indictor of ICU qulity in U.S. ICU popultion. We confirmed tht the FS-ICU mesures two distinct constructs: stisfction with cre nd stisfction with decision mking. Although the instrument s developers envisioned the FS-ICU s mesuring these two domins (5), this is the first study to test this ssumption. Bsed on our nlyses, summry score for the entire FS-ICU ws lso developed. We decided priori tht the numbers of fctors would be determined by prllel nlysis on rndom dt mtrix. Although vrious methods hve been proposed for determining the numbers of fctors, prllel nlysis ppers to be one of the most ccurte (39). The greement by second method (the scree test) dded robustness to our findings nd model interprettion. Our simple scoring method will fcilitte use of the FS-ICU s n outcome mesure in reserch. For exmple, n intervention specificlly trgeting stisfction with cre or stisfction with decision mking might use the pproprite subscle s primry outcome. Despite the utility of the FS-ICU scores, feedbck of individul item performnce still seems to be the most effective wy to identify ctionble res for improvement becuse it gives clinicins specific trgets for improvement (7). This is the first study to vlidte the FS-ICU in U.S. ICU popultion. We demonstrted tht the FS-ICU correltes well with the Fmily-QODD, previously vlidted instrument for mesuring ICU qulity t the end of life (17 21). Overll, we found tht individul QODD items hd higher correltion with the Cre subscle thn the Decision Mking subscle on the FS-ICU. Since the QODD focuses on spects of cre t the end of life, this finding mkes conceptul sense nd lends dditionl credibility to our fctor model nd interprettion of the subscles. Compring the FS-ICU with vrious nurse ssessments of ICU qulity reveled weker ssocitions. In contrst to the Fmily-QODD, the Nurse-QODD did not correlte highly with the FS-ICU nd did 276 Crit Cre Med 2007 Vol. 35, No. 1

Tble 5. Comprisons of the FS-ICU ginst other intensive cre unit (ICU) qulity indictors Stisfction with Cre (FS-ICU/Cre) not rech our cutoff for sttisticl significnce of p.01 (lthough it did chieve p.05). A likely explntion is tht nurses nd fmilies use different criteri for rting the qulity of dying. For exmple, previous reserch with the QODD suggests tht nurses re more criticl of the qulity of end-of-life cre thn fmily members (20). Bsed on the current study s findings, the nurse-ssessed qulity indictors for end-of-life cre pper to mesure construct tht is relted to but lso different from fmily stisfction in the ICU. We tgged items for possible removl bsed on pre-estblished criteri, but the finl decisions regrding retention or removl were mde fter consensus with the instrument s developers. We believed this pproch offered prcticl blnce of sttisticl guidnce nd expert opinion (22). Although our scles incorported only the retined items, others my wnt to use the dropped items for their own evlutive nd qulity improvement efforts. For exmple, spiritul cre nd socil work re importnt items for mesuring the qulity of ICU cre with certin fmilies (2). Therefore, we hve published the originl FS-ICU items long with the shortened 24-item FS-ICU online (12). Stisfction with Decision Mking (FS-ICU/DM) Totl Score (FS-ICU/Totl) Correltion p Correltion p Correltion p Fmily QODD Pin control.488.001.378.001.463.001 Brething comfort.301.002.256.008.317.001 Cre by doctors.578.001.505.001.571.001 Cre by ll providers.693.001.513.001.638.001 Totl QODD score.499.001.517.001.562.001 Nurse QODD Pin control.016.88.128.20.075.455 Brething comfort.045.64.176.07.128.189 Cre by doctors.005.96.174.07.088.366 Cre by ll providers.075.44.122.21.025.794 Totl QODD Score.220.05.248.03.258.023 Nurse brriers Fmily ngry.205.008.252.001.234.003 Fmily unrelistic.169.03.066.4.124.12 Totl number of brriers.195.01.154.05.183.02 Meeting fmily needs Emotionl needs.194.01.259.001.238.003 Fmily communiction.188.02.204.01.201.01 Overll needs met.156.05.215.007.188.02 FS-ICU, Fmily Stisfction in the Intensive Cre Unit survey; QODD, Qulity of Dying nd Deth questionnire. Spermn correltion coefficients for the two FS-ICU subscles nd totl instrument. Crit Cre Med 2007 Vol. 35, No. 1 The current study hs severl importnt limittions. First, the FS-ICU ws designed for generl ICU popultion wheres the QODD is only relevnt for fmily members of ptients who died. In ddition, only hlf of the fmily members in the vlidtion cohort returned questionnire pckets, nd our study includes predomintely white popultion from university hospitl. Although respondent bis my ffect the generlizbility of our results, the internl vlidity of the FS- ICU should not be ffected. Second, we used mrginl medin imputtion for missing dt, nd this is less rigorous pproch thn multiple imputtion. However, the low rte of imputed vlues ( 15% per item) likely minimizes this bis on our estimtes. Third, vlidity testing not only mesures psychometric property of n instrument but lso ssesses the degree to which inferences cn be mde bout the popultion tht produced those results. Since fctor nlysis ws conducted in diverse ICU popultion, our findings my not pply in specilized ICU with less heterogeneous popultion (40). Fourth, we did not evlute test-retest relibility of the shortened instrument nd the originl developers hd only ssessed this property in 25 fmily members (5). Therefore, it is importnt tht future studies reevlute test-retest relibility of the 24-item FS- ICU. Finlly, we developed the FS-ICU scoring system so tht it could be used s n outcome mesure in future studies. It is importnt to remember, however, tht the responsiveness of this instrument hs not been demonstrted. Overll, FS-ICU vlidtion is continuous nd evolving process tht needs to be repeted nd expnded with other popultion smples. CONCLUSIONS Our study suggests tht the FS-ICU is vlid nd relible tool for ssessing fmily stisfction in the ICU. The instrument mesures two min conceptul domins stisfction with cre nd stisfction with decision mking. We reduced the number of items nd incresed the instrument s fesibility for future dministrtion. Tools tht ccurtely mesure qulity re necessry in the ICU for both reserch nd qulity improvement. The usefulness of these instruments depends on their relibility, vlidity, responsiveness, fesibility, nd clinicl prcticbility. Given the importnce of surrogte decision mkers in the ICU, we nticipte tht tools like the FS-ICU will be n essentil prt of the highly functioning ICU s improvement toolbox nd n importnt outcome mesure for rndomized trils. REFERENCES 1. Institute of Medicine. Crossing the Qulity Chsm: A New Helth System for the 21st Century. Wshington, DC, Ntionl Acdemy Press, 2001 2. Clrke EB, Curtis JR, Luce JM, et l: Qulity indictors for end-of-life cre in the intensive cre unit. Crit Cre Med 2003; 31:2255 2262 3. Cohen LM, McCue JD, Green GM: Do clinicl nd forml ssessments of the cpcity of ptients in the intensive cre unit to mke decisions gree? Arch Intern Med 1993; 153: 2481 2485 4. Ely EW, Shintni A, Trumn B, et l: Delirium s predictor of mortlity in mechniclly ventilted ptients in the intensive cre unit. JAMA 2004; 291:1753 1762 5. Heylnd DK, Trnmer JE: Mesuring fmily stisfction with cre in the intensive cre unit: the development of questionnire nd preliminry results. J Crit Cre 2001; 16: 142 149 6. Heylnd DK, Rocker GM, Dodek PM, et l: Fmily stisfction with cre in the intensive cre unit: Results of multiple center study. Crit Cre Med 2002; 30:1413 1418 7. Dodek PM, Heylnd DK, Rocker GM, et l: Trnslting fmily stisfction dt into 277

qulity improvement. Crit Cre Med 2004; 32:1922 1927 8. Heylnd DK, Cook DJ, Rocker GM, et l: Decision-mking in the ICU: Perspectives of the substitute decision-mker. Intensive Cre Med 2003; 29:75 82 9. Lederer MA, Goode T, Dowling J: Origins nd development: The Criticl Cre Fmily Assistnce Progrm. Chest 2005; 128(3 Suppl): 65S 75S 10. Dowling J, Vender J, Guilinelli S, et l: A model of fmily-centered cre nd stisfction predictors: The Criticl Cre Fmily Assistnce Progrm. Chest 2005; 128(3 Suppl): 81S 92S 11. Dowling J, Wng B: Impct on fmily stisfction: The Criticl Cre Fmily Assistnce Progrm. Chest 2005; 128(3 Suppl):76S 80S 12. Fmily Stisfction in the Intensive Cre Unit (FS-ICU) Survey. Avilble t: http:// www.criticlcreconnections.com. Accessed Jnury 1, 2006 13. Johnson D, Wilson M, Cvnugh B, et l: Mesuring the bility to meet fmily needs in n intensive cre unit. Crit Cre Med 1998; 26:266 271 14. Molter NC: Needs of reltives of criticlly ill ptients: A descriptive study. Hert Lung 1979; 8:332 339 15. Wsser T, Psqule MA, Mtchett SC, et l: Estblishing relibility nd vlidity of the criticl cre fmily stisfction survey. Crit Cre Med 2001; 29:192 196 16. Ry DE, Fuhrmn C, Arnold D, et l: Correltion of the qulity of dying nd deth questionnire nd the criticl cre fmily stisfction survey in multi-specilty intensive cre unit. Abstr. Chest 2005; 128:186S 17. Curtis JR, Ptrick DL, Engelberg RA, et l: A mesure of the qulity of dying nd deth. Initil vlidtion using fter-deth interviews with fmily members. J Pin Symptom Mnge 2002; 24:17 31 18. Hodde NM, Engelberg RA, Treece PD, et l: Fctors ssocited with nurse ssessment of the qulity of dying nd deth in the intensive cre unit. Crit Cre Med 2004; 32: 1648 1653 19. Mulrski R, Curtis JR, Osborne M, et l: Agreement mong fmily members in their ssessment of the Qulity of Dying nd Deth. J Pin Symptom Mnge 2004; 28: 306 315 20. Levy CR, Ely EW, Pyne K, et l: Qulity of dying nd deth in two medicl ICUs: Perceptions of fmily nd clinicins. Chest 2005; 127:1775 1783 21. Mulrski RA, Heine CE, Osborne ML, et l: Qulity of dying in the ICU: Rtings by fmily members. Chest 2005; 128:280 287 22. Coste J, Guillemin F, Pouchot J, et l: Methodologicl pproches to shortening composite mesurement scles. J Clin Epidemiol 1997; 50:247 252 23. Nunnlly JC: Psychometric Theory. 2nd ed. New York, McGrw Hill, 1978 24. Bernstein IH, Teng G: Fctoring items nd fctoring scles re different: Spurious evidence for multidimensionlity due to item ctegoriztion. Psychol Bull 1989; 105: 467 477 25. Horn JL: A rtionle nd test for the number of fctors in fctor nlysis. Psychometrik 1965; 30:179 85 26. Cttell RB: The scree test for the number of fctors. Multivrite Behviorl Reserch 1966; 1:246 276 27. Gorsuch RL: Fctor Anlysis. Phildelphi, Sunders Compny, 1974 28. Cronbch LJ: Coefficient lph nd the internl structure of tests. Psychometrik 1952; 16:297 302 29. Wre JE Jr, Kosinksi M, Gndek B: SF-36 Helth Survey: Mnul nd Interprettion Guide. Lincoln, RI, QulityMetric Incorported, 2000 30. Streiner DL, Normn GR: Helth Mesurement Scles: A Prcticl Guide to Their Development nd Use. 2nd ed. Oxford, UK, Oxford University Press, 1995 31. The End of Life Reserch Cre Progrm. Avilble t: http://depts.wshington.edu/ eolcre. Accessed Jnury 1, 2006 32. Wters CM: Professionl nursing support for culturlly diverse fmily members of criticlly ill dults. Res Nurs Helth 1999; 22: 107 117 33. Curtis JR, Ptrick DL, Shnnon SE, et l: The fmily conference s focus to improve communiction bout end-of-life cre in the intensive cre unit: opportunities for improvement. Crit Cre Med 2001; 29(2 Suppl): N26 N33 34. Downey L, Engelberg RA, Shnnon SE, et l: Cring for fmilies of ptients who die in the intensive cre unit: Mesuring nurses perspectives. Am J Crit Cre, In Press 35. Chrles C, Gfni A, Wheln T: Decisionmking in the physicin-ptient encounter: Revisiting the shred tretment decisionmking model. Soc Sci Med 1999; 49: 651 661 36. Truog RD, Cist AF, Brckett SE, et l: Recommendtions for end-of-life cre in the intensive cre unit: The Ethics Committee of the Society of Criticl Cre Medicine. Crit Cre Med 2001; 29:2332 2348 37. Crlet J, Thijs LG, Antonelli M, et l: Chllenges in end-of-life cre in the ICU. Sttement of the 5th Interntionl Consensus Conference in Criticl Cre: Brussels, Belgium, April 2003. Intensive Cre Med 2004; 30:770 784 38. Angus DC: Chrting (nd publishing) the boundries of criticl illness. Am J Respir Crit Cre Med 2005; 171:938 939 39. Zwick WR, Velicer WF: Comprison of five rules for determining the number of components to retin. Psychol Bull 1986; 99: 432 442 40. Delis DC, Jcobson M, Bondi MW, et l: The myth of testing construct vlidity using fctor nlysis or correltions with norml or mixed clinicl popultions: Lessons from memory ssessment. J Int Neuropsychol Soc 2003; 9:936 946 278 Crit Cre Med 2007 Vol. 35, No. 1

Appendix Fmily Stisfction in the Intensive Cre Unit (FS-ICU) survey items Item 1 The courtesy, respect, nd compssion your fmily member (the ptient) ws given. 2 How well the ICU stff ssessed nd treted your fmily member s pin. 3 How well the ICU stff ssessed nd treted your fmily member s brethlessness. 4 How well the ICU stff ssessed nd treted your fmily member s gittion. 5 How well the ICU stff showed n interest in your needs. 6 How well the ICU stff provided emotionl support. 7 How well the ICU stff met your spiritul/religious needs. 8 The temwork of ll the ICU stff who took cre of your fmily member. 9 The courtesy, respect, nd compssion you were given. 10 How well the nurses cred for your fmily member. 11 How often nurses communicted to you bout your fmily member s condition. 12 How well doctors cred for your fmily member. 13 How often doctors communicted to you bout your fmily member s condition. 14 How well the ICU socil workers ssisted nd supported you. 15 How well the ICU chplin ssisted nd supported you. 16 Atmosphere of the ICU. 17 Atmosphere of the ICU witing room. 18 Overll stisfction with your experience in the ICU. 19 Willingness of ICU stff to nswer your questions. 20 How well ICU stff provided you with explntions tht you understood. 21 The honesty of informtion provided to you bout your fmily member s condition. 22 How well ICU stff informed you wht ws hppening to your fmily member nd why things were being done. 23 The consistency of informtion provided to you bout your fmily member s condition. 24 Did you feel included in the decision mking process? 25 Were you involved t the right time in the decision mking process? 26 Did you receive n pproprite mount of informtion to prticipte in the decision mking process? 27 Did you feel you hd enough time to think bout the informtion provided? 28 Did you feel supported during the decision mking process? 29 Did you feel you hd control over the cre of your fmily member? 30 Were you given the right mount of hope tht your fmily member would recover? 31 Ws there greement within your fmily regrding the cre tht your fmily member received? 32 When mking decisions, did you hve dequte time to hve your concerns ddressed nd questions nswered? 33 How stisfied were you with the level or mount of helth cre your fmily member received in the ICU? 34 Overll stisfction with your role in the decision-mking relted to the cre of your fmily member in the ICU. ICU, intensive cre unit. Dropped from survey during item reduction. Crit Cre Med 2007 Vol. 35, No. 1 279