Can Just-in-Time, Evidence-Based Reminders Improve Pain Management Among Home Health Care Nurses and Their Patients?

474 Journal of Pain and Symptom Management Vol. 29 No. 5 May 2005 Original Article Can Just-in-Time, Evidence-Based Reminders Improve Pain Management Among Home Health Care Nurses and Their Patients? Margaret V. McDonald, MSW, Liliana E. Pezzin, PhD, Penny H. Feldman, PhD, Christopher M. Murtaugh, PhD, and Timothy R. Peng, PhD Center for Home Care Policy and Research (M.V.M., P.H.F., C.M.M., T.R.P.), Visiting Nurse Service of New York, New York, New York; and Department of Medicine (L.E.P.) and Health Policy Institute (L.E.P.), Medical College of Wisconsin, Milwaukee, Wisconsin, USA Abstract The purpose of this randomized, controlled, home care intervention was to test the effectiveness of two nurse-targeted, e-mail based interventions to increase home care nurses adherence to pain assessment and management guidelines, and to improve patient outcomes. Nurses from a large urban non-profit home care organization were assigned to usual care or one of two interventions upon identification of an eligible cancer patient with pain. The basic intervention consisted of a patient-specific, one-time e-mail reminder highlighting six pain-specific clinical recommendations. The augmented intervention supplemented the initial e-mail reminder with provider prompts, patient education material, and clinical nurse specialist outreach. Over 300 nurses were randomized and outcomes of 673 of their patients were reviewed. Data collection involved clinical record abstraction of nurse care practices and patient interviews completed approximately 45 days after start of care. The intervention had limited effect on nurse-documented care practices but patient outcomes were positively influenced. Patients in the augmented group improved significantly over the control group in ratings of pain intensity at its worst, whereas patients in the basic group had better ratings of pain intensity on average. Other outcomes measures were also positively influenced but did not reach statistical significance. Our findings suggest that although reminders have some role in improving cancer pain management, a more intensive approach is needed for a generalized nursing workforce with limited recent exposure to state-ofthe-art pain management practices. J Pain Symptom Manage 2005;29:474 488. 2005 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved. Key Words Cancer pain management, provider behavior change, evidence-based medicine, reminders, home care Address reprint requests to: Liliana E. Pezzin, PhD, Department of Medicine and Institute for Health Policy Studies, Medical College of Wisconsin, 8701 Watertown Plank Road, Milwaukee, WI 33226, USA. Accepted for publication: August 25, 2004. 2005 U.S. Cancer Pain Relief Committee Published by Elsevier Inc. All rights reserved. Introduction In 2003, roughly 1.3 million persons were diagnosed with cancer. 1 Pain is the most common symptom of cancer and treatments for cancer. Evidence suggests that cancer pain 0885-3924/05/$ see front matter doi:10.1016/j.jpainsymman.2004.08.018

Vol. 29 No. 5 May 2005 Pain Management in Home Health Care 475 is often undertreated, despite the availability of effective interventions. In a recent National Institutes of Health State of the Science conference on Symptom Management in Cancer, 2 experts acknowledged that although research has produced important new insights into the causes and cures of cancer, efforts to manage the symptoms of the disease and its treatment have not kept pace. One important barrier to effective pain management is the knowledge deficit among health care professionals. 3 Sixty-one percent of the oncologists surveyed by the Eastern Cooperative Oncology Group reported that poor pain assessment knowledge and skills contributed to undertreatment. 4 Nurses knowledge about the treatment of cancer pain has been found to be inadequate in both basic nursing education and graduate-level training. 5,6 The situation does not improve once nurses are in the field. In a 1998 survey of home care nurses, 38% reported receiving only one hour of pain education in their careers and another 34% received up to five hours. 7 Without better education, practicing nurses lack confidence in their pain management skills and are over-reliant on physicians for guidance. 8 Developing effective strategies to influence provider behavior and promote evidence-based care across the continuum of care is an area of critical importance. Passive approaches, such as the distribution of consensus recommendations and guidelines, as well as traditional continuing education approaches, increasingly are recognized as ineffective ways of influencing clinical practice. 9 13 More successful strategies are audit and feedback, reminder systems, and in some settings, use of multi-faceted interventions. 14 16 This article describes the study of two computer-based reminder interventions designed to promote evidence-based pain management practices among home care nurses, strengthen their collaborative role in helping patients adhere to physician-prescribed medical and behavioral regimens, and improve patient outcomes. It also examines the relative cost-effectiveness of the two interventions. Justin-time reminder systems push evidencebased guidelines out to the clinician at the right time and place. 17 They are thought to work because they address the dual task theory of human performance, which suggests that presenting specific information to a clinician at a critical time can help improve secondary tasks in conjunction with primary medical care already being provided. 18 Home care nurses usually are generalists with varying educational backgrounds and experiences who serve a diverse group of patients. Cancer patients are referred from hospitals for post-acute care or from community physicians for a variety of services, including close monitoring of disease symptoms and side effects of treatment, post-surgical wound care, and medication and diet education. Most patients in home care present with other co-morbidities that the nurse is addressing as well. As a result, pain is often relegated to a secondary status among the tasks the nurse is performing. Electronic reminders have proven to be effective in other settings, with the interventions targeting specific behaviors such as preventive care, prescribing practices, and test ordering. 16,18 The potential utility of reminders was of special interest in this study setting because of the need to reach a highly decentralized, mobile workforce in a direct, timely, and efficient manner. To the best of our knowledge, this is the first randomized study using reminders to improve pain management in the decentralized home care system, where nearly 9% of over 7 million patients discharged in 1996 had a primary diagnosis of cancer. 19 The study addressed three main questions: 1) whether intervention group nurses would exhibit improved pain assessment and management practices compared to usual care nurses, 2) whether patients served by intervention group nurses would have better symptom management and quality of life than patients served by control group nurses, and 3) which of the two alternative strategies would be more cost-effective. We hypothesized that nurses receiving pain management recommendations at the time they are assigned a patient experiencing pain would be more likely than others to provide evidenced-based pain management care. In addition, we theorized that patients whose nurses received the reminders would have better clinical outcomes. Finally, we expected that care provided by intervention nurses would be more costly, but ultimately more cost-effective, than care provided by control group nurses.

476 McDonald et al. Vol. 29 No. 5 May 2005 Methods Study Design The study employed a randomized design. Each nurse was randomly assigned to either the control group or one of two treatment groups (basic intervention or augmented intervention) the first time s/he began caring for an eligible patient. Randomization occurred after the admitting nurse had completed the initial patient assessment and electronically transmitted assessment data from a portable computer to the agency mainframe. A computerized algorithm developed by project staff identified nurses whose patients met the study criteria. A nurse s initial random assignment to a specific group (usual care, basic intervention, or augmented intervention) determined the status for all new patients allocated to that particular nurse s care for the duration of the study. Although nurses were randomly assigned to treatment or control groups, random assignment of patients to nurses was not feasible. Patients referred to the study agency, however, are assigned to a specific nurse based primarily on where the patient lives and the nurse s overall caseload. Although not random, this assignment process is based on observable and exogenous factors that were subsequently controlled for in the analyses. Furthermore, agency staff responsible for assigning patients to nurses were blinded to the study. Interventions The same basic e-mail reminder was sent to nurses in both intervention groups. The nurses in the intervention groups received the e-mail reminder every time an eligible cancer patient with pain was admitted to his/her care. The e-mail identified the patient by name, indicated that the patient reported pain at admission, and highlighted six clinical pain management practice recommendations for improving patient outcomes. A modified Delphi technique was used to determine the choice of practices to highlight in the e-mail. 20 An expert panel, informed by the results of a literature review and side-by-side comparison of pain management clinical guidelines, identified the most important pain management practices for the home health setting and made suggestions for additional materials to be used in the augmented intervention arm. In addition to the study investigators, the expert panel included nurse pain specialists from two large cancer specialty centers, a hospice physician and nurse, and home care nurse consultants. The study team synthesized the recommendations and created the e-mail reminder that consisted of an initial screen listing six key practices in very abbreviated form (with the first letter of each practice spelling out the acronym RELIEF ), as well as subsequent screens that the nurse could consult for more detailed information (see Appendix 1). The augmented intervention substantially expanded the information and resources available to the nurse. In addition to the basic e-mail, it included a laminated pocket card that directed the nurses how to complete a comprehensive pain assessment, including a 0 10 visual scale to use with patients, a prompter card to help nurses improve communication with physicians, a self-care guide to review with patients and open a dialogue about pain control, and follow-up by an oncology Clinical Nurse Specialist (CNS) who served as an expert peer. The CNS was employed by the agency and available to all staff requesting a consultation. The augmented intervention outreach was a more pro-active approach. It consisted of an e-mail sent by the CNS to the nurse a week after the first e-mail and reminded the augmented group nurse that the CNS was available for consultation. All e-mail reminders and information packets were distributed within 10 days of each new eligible patient s admission to home care. Control group nurses did not receive any intervention materials and provided usual care. Study Population The study population consisted of home health nurses working for a large, certified, nonprofit, urban home health agency and the eligible cancer patients they served. Eligible patients were those aged 18 or older who were admitted with a primary diagnosis of cancer (ICD9- CM140-239) and self-reported frequency of daily or constant pain at admission. Patients who were not cognitively able to give informed consent (as determined by the administration of the short portable mental status questionnaire 21 ), as well as non-english or non- Spanish speaking subjects, were excluded from the study. The study was approved by the appropriate Institutional Review Boards.

Vol. 29 No. 5 May 2005 Pain Management in Home Health Care 477 Data Sources and Variable Definitions The three main questions addressed by the study relating to nurse practices, patient outcomes, and cost-effectiveness of the interventions were addressed through a combination of data collected from record review abstractions and patient interviews. Additionally, the study used data from the Outcomes Assessment and Information Set, or OASIS, a standardized home care assessment instrument completed at every admission, as well as administrative data routinely collected by the agency s billing and human resources departments. Intervention cost logs were maintained by study staff. Baseline measures of patients health and functional status, such as limitations in activities of daily living (ADLs) and instrumental activities of daily living (IADLs), pain frequency and intensity, cognitive functioning, and the presence of certain medical conditions, were derived from the nurse s OASIS assessment conducted during the initial visit. OASIS data also were the source of information on patients sociodemographic and social support characteristics. These data were available electronically and were used for patient screening purposes, attrition analysis, and for case-mix control. Human Resources data were the source of information on the provider nurses baseline control characteristics (e.g., sex, race/ ethnicity, educational level, employment status). Administrative data provided information on the nurse caseload and the reimbursement environment (pre-prospective Payment System or otherwise) and home care service use during the study period. Nurse Practice Measures. Nurse practice measures were obtained from patient clinical records using a structured chart abstraction instrument designed to capture process of care information during a 45-day period post-index admission. Record abstractions were completed by trained nurse reviewers who were blinded to the intervention group assignment of study nurses and their patients. Reviewers were provided with a manual identifying the specific practices that had to be documented to meet each survey item. Trial abstractions were completed by each reviewer and compared to a gold standard review. Additional instruction was provided as needed, and random checks were completed throughout the data collection period to maximize consistency among reviewers. The chart abstraction tool collected information on the nurse s pain assessment of the patient and his/her instructions to the patient concerning pain and its management. The specific assessment and instruction items studied were those identified in the e-mail reminder sent to nurses in the intervention groups. At each home health visit provided by a study nurse to an eligible pain patient during the follow-up period, data were collected on whether the nurse recorded the assessment and instructions detailed below. Assessment Measures. All patients in this study reported pain at the intake visit. For each patient, reviewers determined whether at each subsequent visit completed by the study nurse the chart contained documentation that the nurse assessed: (1) the presence or absence of pain and, for those with pain, (2) the intensity of pain using a numeric rating scale, (3) the intensity of pain using descriptive words, (4) the location of the pain, (5) characteristics of the pain (e.g., throbbing, shooting), (6) aggravating or relieving factors, (7) pain effect on physical and social functioning, (8) medication management, (9) bowel movement regularity, and (10) mood. Assessment of the presence or absence of pain was examined with two binary measures. The first variable was assigned a positive value if the nurse re-assessed pain at least once after the intake visit for each of her patients. In addition, because the intervention promoted pain as the fifth vital sign, a second more stringent indicator was constructed and assigned a positive value if the nurse assessed pain at all visits provided to all of her patients. The remaining assessment and instruction items, which may need to be completed only once for each patient, were defined as binary measures with a positive value if the nurse assessed the item at least once for each of her pain patients in the study. Documentation of pain intensity using a numeric rating scale was a binary measure with a positive value if the nurse recorded intensity using a number for each of her patients at least once during the observation period. Location of pain, medication management, bowel movement assessment, and mood assessment were handled in

478 McDonald et al. Vol. 29 No. 5 May 2005 the same way. Other pain assessment items were combined (pain intensity using descriptive words, characteristics of pain, aggravating or relieving factors, pain effect on physical and social functioning) with a positive value if the nurse recorded any of these pain characteristics for each of her patients at least once during the 45-day follow-up period. Instruction Measures. For each patient in the study, reviewers determined whether the chart contained documentation that the nurse instructed the patient (or an informal caregiver) about: (1) medication management (if taking medication), (2) side effects of medication (if taking medication), (3) the use of complementary therapies, (4) notifying a health professional if pain is not relieved, (5) bowel movement regimen, and (6) other pain management instruction. In addition, we measured whether the nurse documented that (7) she gave the patient any pain management educational material. Due to low frequency of occurrence, instructions for bowel movement regimen, use of complementary therapies, and other miscellaneous instructions were combined into one variable of other pain management instruction. All other instruction items are binary measures, indicating whether or not the nurse recorded that s/he gave the instruction at least once to each of his/her patients in the study. Patient-Level Outcome Measures. Measures of patient-level outcomes were derived from the in-person survey designed to capture patient information 45 days post home care admission. In-home interviews based on a structured instrument elicited detailed information on the patient s clinical and functional status, including frequency and intensity of pain, and health care utilization during the follow-up period. All interviews, which took place over an 18-month period, were conducted by trained interviewers blinded to the study groups. Whenever possible, standardized measures and constructs were used in the survey. The Brief Pain Inventoryshort form (BPI) was used to measure pain intensity and pain interference. 22 This instrument asks patients to rate pain intensity during the prior week ( pain at its worst, pain on average ) using 0 to 10 numerical rating scales. Patients also rate pain interference with various aspects of functioning and well being (e.g., sleep, ability to walk, mood, etc.) on 0 to 10 numeric rating scales, which are averaged to provide an overall measure of pain-related functional interference. In addition, the European Organization for Research and Treatment of Cancer (EORTC) questionnaire used to assess health-related quality of life 23 contains a separate pain scale. For the EORTC, binary variables were created to indicate whether patients experienced severe symptoms based on their ranking at the top quartile of the distribution for each of three EORTC scales pain, insomnia and constipation. We also categorized those who were in the top quartile of quality of life (the EORTC scales range from 0 100 with higher values indicating worse outcomes on the symptom subscales and better outcomes on the global health status measure). To assess patient-related barriers to pain management, seven items originally validated in the Barriers Questionnaire by Ward and colleagues 24 and used in the Patient Outcome Questionnaire developed by the American Pain Society, 25 were included in the questionnaire. Finally, we utilized the Pain Management Index (PMI) to measure adequacy of analgesic therapy. The PMI compares the potency of analgesics prescribed with the severity of pain intensity reported by the patient. 26 Additional items, such as assessing use of complementary therapies, were developed specifically for this study by the investigators in consultation with clinicians and providers. Cost Measures. Health care utilization data for cost estimates were obtained through a combination of the agency s administrative records and self-reported data on medical care use collected as part of the patient interview. Two measures of cost (home-care-related and overall health care costs) were examined. Home-care costs included administrative costs (i.e., the incremental cost of implementing the interventions, such as the cost of producing and distributing educational materials, and the cost associated with the consultant clinical nurse specialist) and costs associated with care provision (i.e., direct and indirect costs associated with the provision of home care by nurses, therapists, and home health aides). Overall costs included, in addition to the home care costs just

Vol. 29 No. 5 May 2005 Pain Management in Home Health Care 479 described, resource costs associated with utilization of other health care services: specifically, the cost of hospital and emergency department (ED) services and physician visits during the study period. All medical services (e.g., inpatient nights, ED visits, RN visits) were valued using the average Medicare payment (or provider charges) for each type of service based on published Centers for Medicare and Medicaid Services data. Statistical Analysis Estimates of treatment-control group differences (for both interventions) were generated based on multivariate regression models. In estimating the effects of the interventions on nurses practices, we capitalized on the study s randomized design and estimated impact models that exploit the orthogonality of the treatments to baseline variables to obtain unbiased estimates of intervention effects. Pre-intervention measures of the health and functional status of each nurse s patients, as well as nurse-specific and environmental characteristics that might confound the relationship between interventions and nurses practices were included to control for chance differences across nurses and to improve the precision of estimates of intervention effects. In addition to the main variables of interest membership in the basic or augmented treatment group all regressions included pre-intervention, aggregate measures of pain frequency and intensity, age, level of disability, and number of comorbid chronic conditions of each nurse s patients; the nurse s age, sex, race/ethnicity, educational level, experience, employment status, and overall caseload; and county of practice. Analyses of patient-level outcomes followed the basic multivariate model described above. A patient s membership in the usual care, basic or augmented group, however, although based on exogenous factors, was not random. For this reason, differences in outcomes across patients treated by nurses randomly assigned to each of these three groups cannot be thought of as pure intervention impacts. Multivariate analyses that control for patients baseline characteristics nonetheless provide important insights into the effects of the interventions on patient outcomes. In particular, we controlled for patient-level baseline measures of health and functional status assessed by the nurse during the initial visit, including frequency and intensity of pain, ADL and IADL limitations, limitations in cognitive functioning, and the presence and number of certain pre-existing medical conditions; the patient s sociodemographic characteristics, including age, sex, race/ethnicity, marital status, education, expected payment source, and baseline measures of social support; the provider nurse s baseline characteristics; and an indicator of the reimbursement environment (pre-prospective Payment System or otherwise). Finally, measures of the nurse s caseload at the time of patient assignment and county of practice were also included to control for factors simultaneously affecting patient assignment to a specific nurse and patient outcomes. Sample Attrition. A standard concern that arises in experiments with longitudinal follow-up is the potentially biasing effects of sample attrition. Two conditions are necessary for the intervention estimates to be biased: (i) the pattern of attrition differs by treatments and controls and (ii) attrition is correlated with the outcome measure being examined. Because there was evidence of differential attrition among the groups in the study, we followed the traditional econometric approach of estimating outcome models jointly with a sample retention equation to produce attrition-corrected estimates of the interventions on nurse process measures and patient outcomes. Specifically, a bivariate probit specification was used to model process and outcome measures that were binary in nature (yes/no) while a two-stage Heckman selection correction specification was used to model continuous outcome variables. 27 The parameter estimates of the models estimated with attrition correction were the basis for the results reported below. In addition, standard errors were adjusted in all patient-level analysis to account for design clustering (i.e., multiple observations on patients for a given nurse) and heteroskedasticity effects. The magnitude of the intervention effects was estimated by comparing attrition-corrected, regression-adjusted outcome probabilities for the three intervention groups. Specifically, the regression equation for each process or outcome measure was used to calculate adjusted outcomes/process indicators for each individual assuming first treatment (e.g., augmented intervention) and then control status,

480 McDonald et al. Vol. 29 No. 5 May 2005 holding the other variables constant. The average of the individual-level predicted values for each outcome/process indicator represents the regression-adjusted probability in the presence and absence of each intervention. Multivariate methods similar to those described above were used to obtain estimates of attrition-corrected adjusted treatment effects on (the logarithmic transformation of) homecare-related and overall costs. All data analyses were conducted using SAS 8.2 and Stata 7.0 statistical software. Results Nurse Characteristics The characteristics of the nurses assigned to the control and the two intervention groups (i.e., basic and augmented) are reported in Table 1. Overall, the nurses in the study were female (94.6%) with an average age of 43.3 years. A little more than 60% of the nurses were black, non-hispanic, 26.2% were white, non- Hispanic, and 6.6% were Hispanic. Most nurses were per diem employees (i.e., 62.5% were paid on a per visit basis) and had been employed by the home care agency for an average of 6.8 years. Slightly more than half the nurses had a bachelor s degree (53.0%) and an additional 4.5% had an advanced degree (master s or higher). The majority of study nurses cared for three or more eligible pain patients (57.7%). There were no statistically significant differences between control and basic intervention nurses or between control and augmented intervention nurses. Patient Characteristics Of the 1729 patients meeting study criteria, 445 (25.7 %) were found to be ineligible for the survey during the screener telephone interview due to death or institutionalization. Of the 1284 eligible respondents, 238 (18.5%) 87 (18.3%) in the basic group, 60 (15.7%) in the augmented group, and 91 (21.2%) in the control group could not be located or had moved out of the area at the 45-day follow-up. An additional 373 (29.1%) 145 (30.6%) in the basic group, 124 (32.5%) in the augmented group, and 104 (24.2%) in the control group refused Table 1 Characteristics of the Nurse Populations (n 336) Usual Care Basic Intervention Augmented Intervention (n 118) (n 121) (n 97) Percent female 94.1% 95.0% 94.9% Age Mean age in years (SD) 42.8 (9.6) 43.4 (9.4) 43.7 (8.8) Age in categories, % 36 25.4 23.1 21.7 36 45 37.3 39.7 34.0 46 55 23.7 27.3 35.1 55 13.6 9.9 9.3 Race/Ethnicity, % Black, non-hispanic 61.9 59.5 49.5 White, non-hispanic 21.2 24.0 35.1 Hispanic 6.8 8.3 4.1 Other or unknown 10.2 8.4 11.3 Percent per diem 64.4 59.5 63.9 Mean years of employment (SD) 6.8 (5.9) 6.8 (5.7) 6.7 (5.8) Educational level, % Diploma 12.7 9.1 14.4 Associate 25.4 28.9 21.7 Bachelor 51.7 54.6 52.6 Advanced degree 6.8 2.5 4.1 Missing 3.4 5.0 7.2 Number of eligible patients, % 1 22.9 15.7 26.8 2 25.4 17.4 19.6 3 21.2 22.3 18.6 4 or more 30.5 44.6 35.1 Comparisons were made between the control group nurses and the basic group nurses and the control group nurses and the augmented group nurses. There were no statistically significant differences.

Vol. 29 No. 5 May 2005 Pain Management in Home Health Care 481 to be interviewed. Complete interview data were, therefore, available for 673 patients (52.4%) 242 (51.0%) in the basic group, 197 (51.7%) in the augmented group, and 234 (54.5%) in the control group. Our attritioncorrected analysis of the effectiveness and costeffectiveness of the two interventions focuses on these 673 subjects. Table 2 presents selected sociodemographic and baseline health characteristics of the study patients, by intervention group. There were no marked differences in mean age, race, or sex among the three groups. The mean age of the interviewed sample was approximately 63 years old; 9.4% were 45 or younger. About 29.4% of all patients were black, non-hispanic, and nearly 33.0% were of Hispanic descent. Selfreported pain intensity rating, as reported by the patient during the nurse s pre-intervention assessment visit, was 5.4 (SD 2.1) and similar across groups. The groups differed, however, with respect to whether the patient had surgery immediately prior to home care admission and time since cancer diagnosis. Patients in the basic group were more likely than those in the control group to have had surgery prior to the index admission (58.3% compared to 44.9%). Patients in the augmented group were more likely than usual care patients to have been diagnosed within the past 6 months (55.3% compared to 49.1%). Finally, there were no significant differences across groups with respect to physical functioning, as measured by ADL and IADL limitations, or number of comorbidities. Impact of the Interventions on Nurse Practices To evaluate if nurses in the intervention groups used more evidence-based pain management practices as compared to the usual care group, nurses assessment and instruction practices regarding pain management were abstracted from clinical charts based on a structured abstraction tool. Over 85% of nurses across all groups reassessed pain at least once for each of their patients who reported pain at admission; over 75% documented the location of the patient s pain; and slightly more than 50% Table 2 Key Sociodemographic and Baseline Health Characteristics of Study Patients (n 673) Usual Care Basic Intervention Augmented Intervention (n 234) (n 242) (n 197) Age Mean age in years (SD) 62.9 (13.3) 63.2 (13.0) 63.4 (12.4) Age in categories, % 45 12.8 7.9 7.1 45 54 11.5 14.9 17.3 45 64 26.5 26.0 26.4 65 74 26.5 29.8 26.9 74 22.7 21.5 22.3 Percent female 64.5 68.6 65.5 Race/Ethnicity (%) Black, non-hispanic 30.8 26.5 31.5 Hispanic 33.3 34.3 31.0 White, non-hispanic 29.9 34.7 32.0 Other or unknown 6.0 4.6 5.6 Time since Dx in years, mean (SD) 2.4 (4.5) 2.3 (4.6) 1.5 (2.2) a Length of time since Dx in categories, % 6 months or less 49.1 54.5 55.3 7 12 months 14.1 12.8 11.7 More than one year 35.0 28.9 31.5 Pre-intervention frequency of pain: constant pain, % 13.2 16.5 10.7 Pre-intervention pain intensity rating (range: 0 10) 5.3 (2.2) 5.4 (2.1) 5.4 (2.2) Surgery pre-admission, % 44.9 58.3 a 52.3 Primary Dx Severity Rating, mean (SD) b 2.4 (0.8) 2.5 (0.8) 2.4 (0.8) Co-morbidity Score, mean (SD) c 1.3 (1.4) 1.5 (1.5) 1.2 (1.2) ADL/IADL Score, mean (SD) b 5.5 (2.7) 5.5 (2.7) 5.5 (2.5) Comparisons were made between the control group nurses and the basic group nurses and the control group nurses and the augmented group nurses. a Indicates differences that are statistically significant at the p 0.05 level. b Scores and ratings were based on a standardized start of care assessment data completed by a home care nurse blinded to study group. Higher values indicate greater disability. c Score was calculated using interviewer collected data.

482 McDonald et al. Vol. 29 No. 5 May 2005 documented at least one other characteristic of the pain for each patient. Numeric intensity ratings were documented for each of their patients by about 28% of the nurses. There were no statistically significant differences between the usual care group nurses and the intervention group nurses in these measures. Basic intervention group nurses completed mood assessments more frequently than usual care nurses (92.7% vs. 85.5%, P 0.08), but completed bowel movement assessments at a lower frequency (89.0% vs. 94.7%, P 0.02). Documentation of pain management instruction was fairly low. Medication management instruction for each of their patients taking medications for pain was reported by 30.7% to 34.7% of the nurses, although augmented group nurses had a higher frequency of reported patient instruction on medication side effects then usual care nurses (21.4% vs. 11.7%, P 0.07). Augmented group nurses were also more likely than control group nurses to distribute pain management educational material to each of their patients (7.3% vs. 1.3%, P 0.07). There were no significant differences between control and intervention groups, however, on the frequency of other pain management instructions or the frequency of nurses telling each of their patients to contact their physician if pain continued or got worse (Table 3). Impact of the Interventions on Patient-Level Outcomes Table 4 reports the regression-adjusted effects of the interventions on patients pain-related outcomes. We tested to see if patients treated by intervention group nurses had better clinical outcomes as compared to patients treated by usual care nurses. Overall, patients in the usual care, basic, and augmented intervention groups had mean scores for pain at its worst of 4.5, 3.6, and 3.4, respectively (scores range from 0 to 10 with lower scores representing better outcomes). Although large in magnitude, the 0.9 point reduction (20.0%, P 0.13) in pain at its worst for the basic intervention patients did not achieve statistical significance at conventional levels. However, the marked 1.2 point reduction (26.7%) in pain at its worst among patients treated by nurses randomized to the augmented intervention, compared to patients receiving usual care, was statistically significant (P 0.05). Adjusted scores for pain on average were 3.7, 2.2, and 3.1, for patients in the usual care, basic, and augmented groups. Patients in the basic intervention group Table 3 Estimates of Treatment Effects on Nurse Practices Usual Care Basic Intervention Augmented Intervention Adjusted Adjusted Adjusted Probability Probability Difference Probability Difference Nurse assessment practices Presence of pain 86.9 89.3 2.4 [0.57] 88.0 1.1 [0.81] Presence of pain at every visit 35.0 39.0 4.0 [0.63] 38.1 3.1 [0.53] Pain intensity (using numeric scale) a 26.2 31.9 5.7 [0.39] 27.9 1.7 [0.80] Location of pain a 82.3 76.8 5.5 [0.35] 82.4 0.1 [0.99] Other assessments of pain a 54.3 60.6 6.3 [0.38] 54.8 0.5 [0.94] Medication assessment b 44.5 45.6 1.1 [0.86] 50.4 5.9 [0.39] Mood assessment 85.5 92.7 7.2 *[0.08] 88.9 3.4 [0.48] Bowel movement assessment 94.7 89.0 5.7 **[0.02] 92.0 2.7 [0.26] Nurse instruction practices Medication management 30.7 34.7 4.0 [0.50] 31.9 1.2 [0.84] Side effects of medications 11.7 10.3 1.4 [0.74] 21.4 9.7 *[0.07] Other pain management instructions 13.9 16.1 2.2 [0.64] 8.5 5.4 [0.21] Instruction on contacting MD 8.6 7.3 1.3 [0.73] 10.8 2.2 [0.61] Educational materials 1.3 2.4 1.1 [0.59] 7.3 6.0 *[0.07] p-values in brackets. Statistically significant differences at the p 0.05 and 0.5 p 0.10 are indicated by ** and *, respectively. Adjusted probabilities are calculated based on underlying coefficients from multivariate models that control for the same set of nurse, patient, and location characteristics. The control variables include sociodemographic characteristics of the nurse (age, sex, race/ethnicity) as well as the nurse s home care employment status and experience, educational level, and caseload; the average baseline characteristics of the patients cared for by each nurse including pain frequency and intensity, the health and functional status of the patients; and the geographical area where the nurse provided care. See Methods for details on the variables reported. a Assessed among patients who reported having pain. b Assessed among patients who were taking an analgesic.

Vol. 29 No. 5 May 2005 Pain Management in Home Health Care 483 Table 4 Estimates of Treatment Effects on Patient Outcomes Usual Care Basic Intervention Augmented Intervention Adjusted Adjusted Adjusted Probability/Score Probability/Score Difference Probability/Score Difference Pain Pain at its worst (range: 0 10) 4.5 3.6 0.9 [0.13] 3.3 1.2** [0.05] Pain on average (range: 0 10) 3.7 2.2 1.5** [0.03] 3.1 0.6 [0.42] Pain interference scale (range: 0 10) 5.3 5.8 0.5 [0.11] 5.2 0.1 [0.86] EORTC a Best quality of life (scale 74) 16.1% 16.9% 0.8 [0.79] 15.2% 0.9 [0.81] Severe pain (scale 74) 28.4% 32.0% 3.6 [0.44] 25.8% 2.6 [0.57] Severe insomnia (scale 74) 40.9% 39.5% 1.4 [0.79] 32.8% 8.1 [0.15] Severe constipation (scale 74) 18.9% 14.8% 4.1 [0.27] 12.0% 6.9* [0.08] Symptom management Inadequate pain management 68.5% 69.9% 1.4 [0.70] 64.0% 4.5 [0.29] Barriers summary score 37.7 37.6 0.1 [0.98] 39.0 1.3 [0.81] Use of alternative treatments 26.9% 22.6% 4.3 [0.22] 15.9% 11.0** [0.02] p-values in brackets. Statistically significant differences at the p 0.05 and 0.5 p 0.10 are indicated by ** and *, respectively. Adjusted probabilities are calculated based on underlying coefficients from multivariate models that control for patient, nurse, and location characteristics. These control variables include socio-demographic characteristics of the patient (age, sex, race/ethnicity, marital status, education, expected payment source); baseline measures of patients health and functional status, including frequency and intensity of pain, limitations in ADLs and IADLs, physical, social, and cognitive functioning, and presence of comorbidities; baseline measures of patients social support and living arrangements; the provider nurse s baseline characteristics, including sex, home care experience, educational level, and caseload; and environmental characteristics such as county of residence. a Binary scores created to indicate patients in the top quartile of the distribution for each scale; higher score indicates better outcome on Quality of Life scale, worse outcomes on the symptom subscales. experienced significantly lower levels of pain on average than their usual care counterparts ( 1.5 points or 40.5%, P 0.03). Despite reduction in reported pain levels, there were no statistically significant differences in pain interference across the three groups. With the exception of a lower prevalence of severe constipation among augmented intervention patients, the interventions did not have a significant effect on scales of the EORTC. Patients in the augmented group were less likely than their control group counterparts to score in the upper quartile (worse outcomes) of the EORTC constipation subscale (12.0% vs. 18.9%, respectively, a difference of 6.9 percentage points, or 36.5%; P 0.08). The other results for the EORTC subscales generally were in the expected direction. However, the coefficients tended to be imprecisely estimated and did not achieve statistical significance. Similarly, although patients in the augmented group were 4.5 percentage points (or 6.6%) less likely to experience inadequate medication management than their usual care counterparts, suggesting better analgesic control of pain among patients in that treatment group, adjusted differences were not statistically significant. There were also no significant intervention effects on barriers to pain management adherence. Finally, use of complementary therapies, such as massage and relaxation/imagery, which was encouraged in the e-mail reminders, was significantly lower among augmented group patients. Service Use, Cost, and Cost-Effectiveness Regression-adjusted estimates of home-carerelated costs are presented in the top panel of Table 5. The average cost associated with administrative and direct home care provision was US$2642 for the usual care group, $2789 for the basic intervention, and $2903 for the augmented intervention group. Thus, the home care related cost of the basic intervention was about 5.6% higher, and the augmented intervention about 9.9% higher than the comparable cost associated with usual care, although neither difference was statistically significant. There were also no statistically significant differences between the intervention and control groups in total costs. However, the probability of a hospital stay during the followup period was significantly lower for augmented compared to control group patients (16.6% vs. 22.2%, respectively, P 0.08). The interventions cost-effectiveness with respect to two outcomes pain at its worst and probability of hospitalization are shown in the bottom panel of Table 5. A 10% improvement in the pain at its worst scale was achieved at an

484 McDonald et al. Vol. 29 No. 5 May 2005 Table 5 Estimates of Service Use, Cost, and Cost Effectiveness Usual Care Basic Intervention Augmented Intervention Adjusted Adjusted Adjusted Probability/Cost Probability/Cost Difference Probability/Cost Difference Service use Probability of hospitalization, % 22.2 22.1 0.1 [0.97] 16.6 5.6* [0.08] Probability of ED use, % 36.6 37.8 1.2 [0.96] 33.5 3.1 [0.38] Cost measure Home care related costs, US$ 2642 2789 147 [0.55] 2903 261 [0.33] Overall costs, US$ 5687 5966 279 [0.57] 5611 76 [0.88] Cost effectiveness: home care related cost of a 10% reduction in (US$): Pain at its worst -- a 97 Pain on average -- 37 a Probability of hospitalization -- a 466 p-values are in brackets. Statistically significant differences at the p 0.05 and 0.5 p 0.10 are indicated by ** and *, respectively. Adjusted costs are calculated based on underlying coefficients from multivariate models that control for the same set of patient, nurse, and location characteristics. These control variables include sociodemographic characteristics of the patient (age, sex, race/ethnicity, marital status, education, expected payment source); baseline measures of patients health and functional status, including pain frequency and intensity, limitations in ADL and IADL activities, physical, social, and cognitive functioning, and presence of comorbidities; baseline measures of patients social support and living arrangements; the provider nurse s baseline characteristics, including sex, home care experience, educational level, and caseload; and environmental characteristics such as borough of residence. Standard errors have been adjusted to account for clustering effects (i.e., multiple observation on patients for a given nurse). a Intervention not effective in improving this outcome. average of $97 for home-care-related costs under the augmented intervention; the basic intervention was not effective in improving this outcome. Similarly, a 10% reduction in the probability of hospitalization under the augmented intervention was achieved with home-care-related costs of $466. Under the basic intervention, pain on average was reduced by 10% with average home-care-related costs of $37. Discussion The high prevalence of cancer-related pain and its undertreatment have received increasing attention over the past two decades, culminating in statements by the Institute of Medicine 3 and the National Institutes of Health 2 emphasizing that action must be taken to improve the adequacy of treatment. Despite numerous efforts over the past 20 years to (re-)educate health care professionals, improvement in pain management appears to have occurred slowly, with clinical practice changing very little. 5 The study presented here contributes to the literature in several important ways. It examines the use of reminders in a clinical context (pain management) and a setting (home health care) that have not been the focus of prior reminder studies. It goes beyond most other studies of the effectiveness of computer-based decision support systems by examining patient outcomes in addition to changes in care practices. 28 It also adds to our understanding of the relative cost effectiveness of strategies designed to improve patient outcomes in specific organizational settings. This comparative, cost-effectiveness approach by and large has been missing from translational research. 11,16,29 The interventions were successful in achieving our primary goal of reducing pain intensity. Patients treated by nurses in the basic and augmented intervention groups had significantly less intense pain at the 45-day follow-up interview. There was also some indication that patients treated by augmented group nurses had less severe constipation. In addition, patients treated by augmented group nurses had a 25% reduction in the probability of a hospitalization. The cost-effectiveness results indicate that incremental improvements in the level of pain experienced by cancer patients treated by nurses in the interventions can be achieved at moderate costs. These findings highlight the importance of including cost calculations in considering whether and how to implement alternative quality improvement measures. The absence of significant intervention effects on patient outcomes other than pain and constipation merits comment. Furthermore, contrary to our expectations, the results revealed only small and marginally significant practice differences among nurses in the intervention and control groups. Nurses in the

Vol. 29 No. 5 May 2005 Pain Management in Home Health Care 485 augmented group documented better pain management practices in just 2 of the 13 processes reviewed (instructing their patients about the side effects of medications and distributing pain management educational materials), whereas those in the basic group recorded better care on just one practice parameter (mood assessment) and worse on another (bowel movement assessment). Clearly, despite the positive effect reminders seemed to have had on certain patient outcomes, there is still a great deal of room for improvement in both the pain-related symptoms experienced by patients and in documented (and presumably actual) nurse practices. Comprehensive pain assessment is the cornerstone of pain management. While more than 85% of nurses reassessed the presence of pain for each of their patients at least once after the initial assessment and more than 75% recorded the location of pain, thorough repeated assessments were lacking. Even with an explicit reminder that pain should be considered as the fifth vital sign, only 38 39% of the intervention group nurses recorded the presence or absence of pain at every visit for each of their patients, only 28 32% recorded a numeric intensity rating, and documentation of any sort of pain management instruction was provided by only 40 42% of the nurses for each patient. Examining the mechanism of the intervention in relation to one of the leading provider behavioral change models the Precede/Proceed model of Green and Kreuter may help us understand some of the interventions limitations. This model emphasizes the role of predisposing, enabling, and reinforcing factors in influencing change. 30 Predisposing factors refer to a variety of individual practitioner characteristics (such as training, knowledge, and beliefs) that affect motivation to change. Enabling factors include organizational and structural factors (such as reminders, checklists, or information systems) that facilitate behavior change. Reinforcing factors include positive consequences, both tangible and intangible, that reward selected behaviors. The interventions tested here focused primarily on enabling factors. Their common objective was to remind nurses of best pain management practices through the basic e-mails and the augmented materials and peer support. However, given nurses relatively limited prior exposure to state-of-the-art pain management care, the information in the reminders may have been too new or unfamiliar to evoke a powerful response. With a stronger core education component provided in advance of or in conjunction with the reminders, nurses might have been more predisposed and confident to react effectively to the reminders and document their actions. The literature shows that lack of confidence in pain management skills is common among nurses and is a barrier to better care. 8,31 Home care nurses often deal with complex patients who present competing clinical care priorities. The patients in this study were not admitted to home care for the primary purpose of pain management. With limited time and resources, nurses must decide what to achieve at each visit, and comprehensive pain assessment and related instruction may be perceived as lower priority than other care management issues. The information reminders were directed to the field nurse to promote pain as a priority, but the intervention did not attempt to influence more global organizational practices. Although improved pain management recently has emerged as a priority area within the study organization, this was not the case during the study period. Nor did the agency routinely use any measure of improvement in patients pain to assess the performance of nursing teams or individual nurses. Perhaps if additional enabling factors were incorporated in the design, the intervention would have achieved greater effectiveness. For example, a concerted effort to engage direct supervisors in the intervention might have strengthened its impact on nurse practices by demonstrating institutional commitment to improving pain management. Or a more structured effort to strengthen nurse-patient communication might have increased the intervention s impact on the full range of pain-related patient outcomes. The use of reinforcing factors, such as measurement and feedback regarding nurses pain management performance and/or patients pain related outcomes, might also have increased the impact of the reminders. With the advent of national home care quality indicators, which include cross-agency comparisons of improvement in pain interfering with patients activities, at least one outcomes-based pain measure is now