Management and Delivery of Community Nursing Services in Ontario: Impact on the Quality of Care and the Quality of Worklife of Community-based Nurses

Transcription

1 Management and Delivery of Community Nursing Services in Ontario: Impact on the Quality of Care and the Quality of Worklife of Community-based Nurses April, 2004 Diane Doran, PhD, RN Jennie Pickard, MScN, RN Janet Harris, MScN, RN Peter C. Coyte, PhD Andrew MacRae, PhD Heather Laschinger, PhD, RN Gerarda Darlington, PhD Jennifer Carryer, MN, RN Decision Maker Partners: Janet Harris, CEO, Durham Access to Care Funding Provided by: Canadian Health Services Research Foundation Ontario Ministry of Health and Long-Term Care University of Toronto

2 Principal Investigators: Diane Doran, RN, PhD Associate Dean, Research and International Relations Faculty of Nursing University of Toronto 50 St George Street Toronto, Ontario M5S 3H4 Telephone: (416) Fax: (416) Jennie Pickard, RN, MScN Director, Quality, Contracts, Research and Organizational Development Durham Access to Care 209 Dundas St. East, 5 th Floor Whitby, Ontario K1N 7H8 Telephone: (905) , ext Fax: (905) jenniepickard@date.org This document is available on the Canadian Health Services Research Foundation web site ( For more information on the Canadian Health Services Research Foundation, contact the foundation at: 1565 Carling Avenue, Suite 700 Ottawa, Ontario K1Z 8R1 communications@chsrf.ca Telephone: (613) Fax: (613) Ce document est disponible sur le site Web de la Fondation canadienne de la recherche sur les services de santé ( Pour obtenir de plus amples renseignements sur la Fondation canadienne de la recherche sur les services de santé, communiquez avec la Fondation : 1565, avenue Carling, bureau 700 Ottawa (Ontario) K1Z 8R1 Courriel : communications@fcrss.ca Téléphone : (613) Télécopieur : (613)

3 Management and Delivery of Community Nursing Services in Ontario: Impact on the Quality of Care and the Quality of Worklife of Community-based Nurses 1 Faculty of Nursing, University of Toronto 2 Durham Access to Care 3 University of Toronto 4 The Research Institute at Lakeridge Health 5 School of Nursing, University of Western Ontario 6 University of Guelph Diane Doran, PhD, RN 1 Jennie Pickard, MScN, RN 2 Janet Harris, MScN, RN 2 Peter C. Coyte, PhD 3 Andrew MacRae, PhD 4 Heather Laschinger, PhD, RN 5 Gerarda Darlington, PhD 6 Jennifer Carryer, MN, RN Acknowledgements: We gratefully acknowledge Canadian Health Services Research Foundation, the Ontario Ministry of Health and Long-Term Care and the Nursing Effectiveness Utilization and Outcomes Research Unit, Faculty of Nursing, University of Toronto, for their support of this research. The opinions, results, and conclusions are those of the authors. No endorsement by the funders is intended or should be inferred. We would like to thank the Community Care Access Centres, nursing agencies, nurses, and clients who participated in this study. The research team recognizes the time and commitment that this study required from all of these people. We also thank them for being open and accommodating when asked to share sensitive data. We would also like to thank the following research assistants for their contribution: Suzanne Fredericks, Jiao Jiang, Shannon Carey, Katie Fisher, Helen Mann, Tania Arko, and Antonio Valente. Special thanks to Sharon Relova, Barbara Thomson and the Statistical Consulting Service, Dept. of Statistics, University of Toronto. A steering committee provided consultation and advice from the preliminary planning stages of the study. The same organizations have been represented on the steering committee throughout the study, although the names of representatives have changed for some organizations. Comcare Health Services Mary Jo Dunlop, Marg McAllister, Heather Arts, Greg More Ministry of Health and Long-Term Care Susan Chernin, Irene Medcof Ontario Association of Community Care Access Centres Georgina White Steering Committee ParaMed Home Health Care Steve Haas Saint Elizabeth Health Care Shirlee Sharkey Victorian Order of Nurses Sue VanDeVelde-Coke, Diane McLeod

4 Table of Contents Key Implications for Decision Makers... i Executive Summary... ii Study Overview...1 Study Background...1 Background Literature...2 Study Objectives...9 Phase Two Design...10 Sample Recruitment...10 Study Variables...12 Participants...15 Nurse Sample...15 Client Sample...16 Contract Characteristics...17 Study Results Phase Two...18 Approach to Data Analysis, Phase Two...18 Objective Consistency of Care Provider...20 Client Perception of the Quality of Nursing Care...21 Nurses Perception of the Quality of Care...22 Contract Management...24 Objective Client Satisfaction Linear Modeling...27 Objective Objective Perceived Job Security...31 Discussion...32 Study Limitations...32 Discussion...34 Objective Objective Objective Objective Conclusion...40 References...42 APPENDIX A...45

5 Key Implications for Decision Makers This study investigated the effect of the competitive model for awarding homecare nursing services by community care access centres in Ontario on the quality of care, and on nurse and client outcomes. Clients cared for by for-profit agencies reported slightly higher quality of care and higher satisfaction than clients cared for by not-for-profit agencies. Nurses felt quality of care was the same at for-profit and not-for-profit agencies. Most access centres and agency managers believed that quality of care and quality improvement initiatives had either remained the same or improved, although some not-for-profit agencies believed the quality of care has declined. Policy makers should focus on providing opportunities for full-time or regular part-time employment (rather than casual employment), providing employment benefits, and providing nurses with more time to provide care. Clients functioned better on social and emotional levels when more visits were made by registered nurses (as opposed to other classes of nurses). Having clients be consistently seen by the same nurse was associated with lower nursing costs but was not related to client health. The likelihood that clients would be seen consistently by the same nurse did not depend on whether the agency was for-profit or not-for-profit. It was more likely when agencies were awarded longer contracts. Nurses reported moderate work enjoyment and low satisfaction with their time for care and job security. There were no differences based on whether they worked for a for-profit or a not-for-profit agency. Nurses who were compensated for their work on an hourly basis reported higher satisfaction with the time for care than those reimbursed per visit. i

6 Executive Summary Background In 1995, the Ontario Ministry of Health and Long-Term Care created 43 community care access centres, which were charged with the responsibility of awarding service contracts to provider agencies using a competitive request for proposals process. Both for-profit and not-for-profit agencies submit proposals to the centres in response to requests for proposals, and in turn are awarded contracts for pre-determined periods of time. The Community Nursing Services Study described this model and examined its impact on nurse and client outcomes. The study was conducted in two phases. The report of the first phase was released in August It described how the competitive bidding process was being put into operation, including the volumes and costs of nursing visits for for-profit and not-forprofit agencies during the first five years of competitive bidding. The report can be read at Objectives The objectives of the second phase of the study were to examine the relationships between variables in the structure of the contracts and: $ quality of care; $ client outcomes; $ cost of care; and $ nurse outcomes. ii

7 This report describes the second phase of the study. Findings There were few differences in quality of care based on the contract length, contract volume, or whether the agency was for-profit or not-for-profit. Clients cared for by forprofit agencies reported higher quality of care and higher satisfaction than clients cared for by not-for-profit agencies. There were significant differences in nurse quality outcomes among provider agencies. The more consistently the client was seen by the same nurse, the lower the nursing costs were. Consistency did not affect client health outcomes. Clients who were healthier when they were admitted to homecare were still healthier six weeks later or at discharge. More visits made by a registered nurse were related to better emotional and social functional outcomes. Older nurses and nurses who worked part-time enjoyed their work more than younger nurses and those who worked on a casual basis. Nurses who were compensated on an hourly basis reported higher satisfaction with time for care than nurses who were compensated on a per visit basis. Nurse turnover was not related to contract variables or ownership type. Design A longitudinal design was used to collect data on client outcomes twice: at admission to homecare or recruitment into the study; and at discharge or after six weeks, whichever came first. Nurses, agencies, and access centres were surveyed using written questionnaires and data abstraction from corporate databases. iii

8 Setting/Sample The setting consisted of 11 community care access centres and 11 nursing agencies. The sample included 740 clients and 700 nurses. Variables Structural variables included information on client demographics, diagnosis, health status, anticipated duration of service, nurses experience, employment characteristics, agency ownership structure (that is, for-profit or not-for-profit), contract length, and contract volume. Process variables included clients and nurses perceptions of the quality of nursing care, consistency of care provider, and percentage of visits by a registered nurse or a registered practical nurse. Outcome variables included costs of nursing care, client satisfaction with care, client health outcomes, nurse work enjoyment, satisfaction with time for care, perception of job security, and annual nursing turnover. Methods Clients completed a structured questionnaire on admission to the study and again after six weeks (or sooner if discharged). Nurses, access centres, and agencies completed written questionnaires once. Analysis was conducted using hierarchical linear modeling. Implications Whether a nursing agency is for-profit or not-for-profit was not strongly related to the quality of services it delivers. Variations in quality existed among both types of ownership structures. This suggests it is important to develop a better understanding of the management and care practices that are most influential in promoting quality performance and better outcomes for clients in the home healthcare setting. iv

9 Study Overview Nationally and internationally, homecare is an increasingly important component of healthcare services. Various models for managing and delivering homecare services have evolved, but there is little evaluative research to assist healthcare managers and policy makers identify the best models for quality, cost-effective care. This study describes the competitive model (request for proposals) used in Ontario and examines the effect of this model on the quality of care and quality of worklife of community-based nurses. Specifically, the study s objectives focused on identifying 1) the extent of for-profit and not-for-profit involvement in the delivery of homecare nursing services; 2) the methods used to award and monitor contracts; and 3) the relationship between the methods and the quality of client care, client satisfaction, nurse satisfaction, job security, and nurse turnover. The examination of the competitive model for awarding homecare services in Ontario was timely in that the model was still in the early stages of development; evaluation has provided useful information to refine the model. Furthermore, knowledge of the strengths and weaknesses of the model can inform healthcare policy in other parts of Canada. The investigators believe that this research will be of compelling interest to managers and decision makers not just in Ontario but across the country, as governments grapple with the challenges of privatization and human and fiscal resource restrictions. Study Background In Canada, the provinces and territories are responsible for providing homecare services. Publicly funded programs exist in every region, although service delivery varies (Health Canada, 1999). Homecare in Ontario falls under the jurisdiction of the Ministry of Health and Long-Term Care. In 1997, the ministry established 43 community care access centres across the province to provide a single point of access to homecare and long-term placement co-ordination services. By 2002, two of the sites had merged, leaving a total of 42 sites. The centres are statutory non-profit corporations under the Community Care Access Corporations Act, 2002, and are accountable to the ministry through a 1

10 memorandum of understanding. The centres purchase services from providers using a competitive process known as a request for proposals. The request for proposals sets out the centre s requirements in areas such as client services, financial management, and organization (Carefoote, 1998). The model embraces the expectation that competition will promote innovation and care standards while achieving cost-effectiveness. Examination of this model is timely, in that it was in the early stages of development and evaluation when the data were collected. It was important to evaluate the impact of the competitive process on the quality and cost of homecare services for several reasons. First, because competition may work in disparate ways, one consequence may be lower prices, thereby forcing lower wages and benefits on staff, leading to decreased morale and increased staff turnover, which in turn can negatively affect the quality of care (Shapiro, 1997). On the other hand, because the centres are able to establish standards for client services, there is potential for the competitive process to lead to enhanced quality in the management and delivery of homecare services. Third, competition may lead to increased efficiencies, innovation, and higher standards of healthcare delivery (Starr, 1999). The study findings will provide useful information to healthcare managers and policy makers for refining the model, and will inform their understanding of the strengths and weaknesses of the model and the effect of competition on healthcare. Background Literature Privatization is the active and conscious transfer of responsibility from the public to the private realm, involving three main activities; regulation, financing, and production (Lundqvist, 1988). Starr (1989) identified four types of government policies that can shift services from public to private: 1) ending public programs and disengaging from specific kinds of responsibilities. Governments may choose to restrict publicly produced services in volume, availability, or quality, which also may lead consumers toward privately produced and purchased healthcare; 2) privatization may take the explicit form of transfers of public assets to private ownership, through sale or lease of public land, 2

11 infrastructure, and enterprises; 3) instead of directly producing some service, the government may finance private services, for example, through contracting out; and 4) privatization may result from the deregulation of entry into activities previously treated as public monopolies. Ontario has a mixture of these elements. Homecare services that meet eligibility criteria are publicly funded. Clients may purchase additional homecare services through private insurance or direct pay, when perceived or actual service needs exceed those that are funded by the public system. Historically, there has been variability among the community care access centres in how contracts are awarded and monitored, although the provincial health ministry recently instituted a new request for proposals template designed to standardize the procurement of client services by centres. The template was developed in consultation with consumer representatives, service providers, and the centres. Both for-profit and not-for-profit agencies are eligible to compete for contracts. There is very little empirical literature examining the issue of privatizing homecare services. One American study found that non-profit home nursing agencies served more welfare, self-pay, and indigent clients, and made more visits per client than for-profit agencies (Shuster & Cloonan, 1991). Nurses from for-profit agencies also spent less time in direct client care (27 percent) than nurses from non-profit agencies (30 percent). In a review of the literature on competitive bidding, McGuire and Riordan (1993) concluded that government contracting for services can only work when the government has good access to information about the service it is purchasing and when performance expectations and standards are clearly spelled out in the performance contract. McCombs and Christianson (1987) described the experience of the National Long-Term Care Channeling Demonstration project in the United States. Of the 10 projects established, five conducted competitive bidding processes to choose providers of community-based long-term care. Two projects chose only one winning bidder, while the other three projects chose multiple winners. The projects with one winner produced lower bid prices, presumably because lower prices were offset by higher volumes. However, there were increased monitoring and administering costs associated with a single winner. When 3

12 there were multiple winners in each service category there were fewer problems in administering and enforcing contracts. When substandard performance existed, case managers simply transferred clients to another service provider (McCombs & Christianson, 1987). The Massachusetts Department of Public Health employed a form of competitive bidding for mental health services (Schlesinger et al., 1986). Schlesinger et al. (1986) concluded that the contracting process added administrative complexity and that the initial cost savings appeared to be relatively small. To the extent that they existed, they primarily reflected lower wages paid by for-profit, as opposed to public, agencies. However, along with lower wages came higher employee turnover and reduced continuity of care. Christianson (1984) and Hillman and Christianson (1984) described the impact of competitive bidding for indigent medical care in Arizona. Lower bid prices were submitted in areas where larger numbers of competitors were expected. Providers faced great uncertainty because the information used to construct per capita bid prices was unreliable. Shapiro (1997) offered an analysis of the impact of competitive bidding for home personal care services in Manitoba. Although contracting out led to lower service costs, it did so at the expense of lower employee wages, higher staff turnover, and lower job security. In addition, Shapiro reported high administrative costs associated with managing the service contract. Managed competition is a topic of interest in healthcare as systems around the world seek ways to contain escalating costs. Much of the international research on managed competition has focused on the acute care sector and on physician services. In response to the mandatory contracting for homecare professional and personal support services in Ontario, an increasing amount of research focusing on the community sector has developed. The recent literature has offered policy analyses of the implications of managed competition, and has evaluated the impact of the competitive model for awarding service contracts on providers and on costs. Findings from this literature are summarized in the following paragraphs. A few studies have investigated the effect of managed competition on caregivers and provider agencies. Abelson et al. (2004, in press) studied the experiences of clients, 4

13 provider agencies, and care managers in home healthcare in one community. They noted that the implementation of the competitive contracting model has focused attention on improved accountability relationships between purchasers and providers, and has begun to improve accountability for meeting contractual obligations. However, they also noted the emerging benefits might be outweighed by the transaction costs incurred by purchaser and provider agencies as well as the quality of care and continuity concerns raised by individual clients and providers who must establish and build new relationships following the awarding of new contracts and agency transfers. Denton, Zeytinoglu, and Davies (2003) studied occupational illnesses among nurses, personal support workers, and therapists working in clients homes. The study sample also included management representatives and office workers. High levels of stress, burnout, and physical health problems were documented, many of which were deemed to be preventable. The study concluded that restructuring and organizational change were significant factors in decreasing job satisfaction, increasing absenteeism rates, increasing fear of job loss, and propensity to leave. In another line of enquiry, a study by Brega, Jordan and Schlenker (2003) examined the variation in homecare utilization at 44 agencies in eight American states. The sample included Medicare clients aged 65 or older with congestive heart failure or diabetes mellitus. Factors such as functional status, complexity, and agency and market characteristics influenced care practices. Patients with greater functional disability received more frequent visits; patients whose initial assessments were conducted by more experienced care providers had longer lengths of stay; proprietary agencies had significantly higher visit intensity, shorter lengths of stay, and more alternative services; and agencies in high-volume states provided more frequent visits and appeared to provide care in a less parsimonious manner than low-volume states. Whether or not an agency was over or under the Medicare per-visit and per-beneficiary cost limits was predictive of length of stay. Also, agencies that used standardized care plans had significantly longer episode lengths than did agencies without such plans. Such care plans may ensure that appropriate care standards are met, regardless of utilization policies. 5

14 Ontario has experienced many changes with respect to access centres and homecare services. In a policy analysis of the relevant literature, Deber (2002) described several issues that arose with the creation of the centres in 1996 and the use of requests for proposals. Deber submitted that some of the complaints regarding the delivery model and disparity in compensation levels could occur regardless of ownership; however, criticisms have been specifically directed towards for-profit private delivery of care. Such complaints included the challenges related to obtaining data when for-profit providers can control disclosure. Schlaht (2001) acknowledged that several aspects were needed to develop a level playing field for managed competition among agencies. These included establishing clear expectations for performance criteria, making judgments about proposals based on appropriate criteria, and achieving consistency within and among the centres, contributing to the perception of fairness during the request for proposals process. The recent introduction of a standardized request for proposals for all centres in Ontario is consistent with this recommendation. At a macro level, Jiwani (2003) completed a case study about the complexities of accountability in healthcare and how accountability was used to shape homecare practices in Ontario. The study described how market logic led to decentralizing homecare organizations while centralizing power through standardization, leading to widespread changes in practices and cultures among provider agencies. The cost of managed competition and care has been explored by several studies. Browne (2000) acknowledged that although market competition could lower costs, improve efficiency, and enhance the quality and the quantity of service in the short term, it does not have positive long-term effects. He contends quality and efficiency have been redefined and made more measurable but trust has been undermined. He argues that competitive bidding has the goal of making workers, agencies, and centres more accountable to the provincial government, but ultimately continuity of care could be weakened if staff turnover results from contracts that could change every few years. Williams et al (1999) noted concerns in the long-term care sector with capped budgets and increasing demands on centres, coupled with managed competition, which could 6

15 result in a decline in service quality and consumer choice. Cloutier-Fisher (2003) later documented concern that the restructuring of community services and the substantial reinvestment in residential long-term care facilities would result in greater institutional care than homecare for some elderly people. In a publication by the Canadian Centre for Policy Alternatives-B.C. Office (2000), it was noted that research into the costs of privatization is needed. The Hamilton-based Community Care Research Centre, accessible at has committed to collecting and cataloguing research on managed competition in Ontario s homecare sector. The research centre is a partnership of community care agencies and McMaster university researchers from health, social, and management sciences. It is funded by the Canadian Institutes of Health Research. Consistency of Care Provider: Consistency of care provider is explored by several studies. Harju and Woodward (2003) reported findings from a study of requests for proposals, provider agency interviews, and agency benchmarks for continuity of care in nursing and homemaking, including consistency of provider as one element. They documented indicators of consistency at one access centre, including frequency of client visit, presence of a primary nurse, existence and size of the team, existence and size of a back-up team, expected compliance with benchmarks, and time interval over which compliance is measured. The authors identified the need for multiple benchmarks to meet the needs of different client groups; for example, clients requiring long-term service, or short-term but high-frequency visits, have different needs for consistency of care provider. The need for standardization about the way consistency is defined and reported by agencies has been articulated, an area that is complex because care is managed by the access centres and delivered by contracted agencies. Woodward, Abelson, Tedford, and Hutchison (2004) conducted an in-depth qualitative study of continuity in homecare from the perspectives of clients, agencies, case managers, and physicians. They described the multiple factors that can influence the number of different nurses who provide care for an individual client, reporting that communication and consistency of personnel were key vehicles to help ensure continuity. Reid, Haggerty, and McKendry (2002) conducted a 7

16 systematic survey of continuity of care, prepared a discussion paper, and facilitated a two-day workshop about the concept. They defined relational continuity as an ongoing therapeutic relationship between a client and health care provider(s). Informational and management continuity were also studied, along with the range of measurement approaches and tools designed to manage continuity. They advocated that multiple measures are needed to capture all aspects of continuity, and that more emphasis is needed on the concept of continuity from the client s perspective and across organizational and sectoral boundaries. In summary, debate about the comparative performance of for-profit and non-profit home healthcare services is prevalent in the healthcare literature (Rosenau & Linder, 2001). Much of this debate has yet to be informed with the findings of well-designed evaluation studies. Managed competition, and in particular, the competitive model for awarding homecare services in Ontario, has spawned its own debate in the literature. In response, researchers are addressing the gap in our scientific knowledge about the effect of managed competition. Key findings from the recent literature suggest that homecare restructuring has resulted in decreased job satisfaction of home healthcare workers, increased absenteeism, and fear of job loss (Denton et al., 2003); undermined trust (Browne, 2000); and perhaps compromised the long-term continuity of care because of staff turnover (Browne, 2000). Most of the studies have focused on care provider and agency perspectives. Research on the relative performance of for-profit and non-profit home healthcare services is a research priority urgently needed (Rosenau & Linder, 2001), as is greater knowledge of the effect of the Ontario model for awarding homecare services on the quality of care, outcomes, and costs for clients. This study was aimed at addressing this gap in the literature. 8

17 Study Objectives The study was conducted in two phases. The objectives of phase one were to: $ assess the extent of for-profit and not-for-profit involvement in the delivery of homecare nursing services; $ describe the method for contracting professional nursing services within the 43 Ontario access centres; $ describe the method for monitoring service agreements; and $ refine the variables and sampling strategy for phase two. The objectives of the second phase of the study were to examine the relationships between contract structural variables and: $ quality of care (process variables); $ client outcomes; $ cost of care; and $ nurse outcomes. The report of phase one was released in August It described how the competitive bidding process was being operationalized, including the volumes and costs of nursing visits for for-profit and not-for-profit agencies during the first five years of competitive bidding. The report can be accessed at or by contacting the principal investigator. This report describes phase two of the study. The design and methodology for phase two are described in the following section, followed by a presentation of the descriptive results and findings for each study objective. 9

18 Phase Two Design Phase two involved a correlational and repeated measures design. The 42 access centres that participated in phase one were eligible to participate in phase two. Sample Recruitment One access centre was randomly selected from each of the six regions in Ontario. The executive director of the centre was contacted by mail and invited to participate in the study by signing and returning a consent form in a self-addressed stamped envelope. If the executive director declined to participate, another centre was randomly selected from the same region until each region was represented. When it was learned that the actual number of eligible nurses and the number of new client referrals were much lower than originally anticipated, a second centre was added in five of the six regions, again by random sampling, for a total of 11 sites participating in phase two. Following enrolment of the access centres, the chief executive officers of the affiliated provider agencies were contacted and invited to participate by signing and returning a consent form. The provider agencies were invited to attend a meeting to learn about the study and the nature of their involvement if they chose to participate and to answer any questions they might have. One agency, with a single contract, declined to participate, resulting in a total of 11 agencies with 34 contracts represented in phase two. Individual respondents for phase two included registered nurses, registered practical nurses, and clients. Nurses were eligible to participate if they had worked for a minimum of six months with a provider agency that consented to participate in the study. Full-time, part-time, and casual nurses were eligible to participate. Clients were eligible to participate if they read English, had been referred to the active caseload of a contracted nursing agency, and consented to participate in the study. A sample size of 700 nurses/clients was sought, based on an estimated small effect size (small (0.03) change in R 2 ), a power of 95 percent, and a significance level of.003 (adjusting for multiple tests of 10

19 significance). The high power and thus the large overall sample size was required in order to allow for an appropriate effective sample size resulting from the cluster sampling used in data collection. Nurse Recruitment: Nursing provider agencies were asked to distribute questionnaires to nurses by preparing two mailing labels for each eligible nurse. An agency representative advised the research co-ordinator of the number of eligible nurses, and the co-ordinator provided packages that contained the invitation to participate, a questionnaire, and a selfaddressed stamped envelope to return the questionnaire to the university research team. Agencies were asked to send a package by mail (postage paid) to every nurse who had been employed with their agency for six months or longer. In order to protect nurse confidentiality, nurses were not identified by name. Return of a completed questionnaire indicated consent to participate in the study. After approximately two weeks, the agencies were asked to distribute a one-page flyer to the same nurses, thanking those who had already returned their questionnaires, and reminding others that a response would be valued by the research team. Client Recruitment: Clients were recruited through the assistance of the access centre s staff. A telephone script was provided for staff to ask new clients if they would consent to have their name and contact information released to the research team. Centres submitted the names of consenting clients to the study co-ordinator, who phoned clients to confirm eligibility and then mailed a letter of explanation, two copies of the consent form, and questionnaire. Clients were asked to return a signed copy of the consent form to the researchers at the University of Toronto address. They were asked to return the completed questionnaire in a separate postage-paid envelope. Clients were also given the option of completing the questionnaire by telephone if this was more convenient, and a toll-free telephone number was provided. A second mailing was sent after approximately three weeks to clients who had not responded to the first mailing. The referral process continued until the required sample had been enrolled. 11

20 Study Variables Table 1 summarizes the study variables. The table is organized in three columns, with the client, nurse, agency, and contract structural variables identified in the left hand column, the care process variables in the centre column, and the client, nurse, and cost outcome variables in the right hand column. Table 1: Study Variables Input (structural variables) Process Variables Outcome Variables Client characteristics Age Gender Marital status Live alone (yes/no) Change in health status (in past week) Anticipated duration of service Cancer diagnosis (yes/no) Diagnosis category Telephone screener, general activities of daily living (ADL) Telephone screener, instrumental activities of daily living (IADL) Client-perceived quality of nursing care Nurse-perceived quality of nursing care Consistency of care provider Percentage of visits by a registered nurse Client Outcomes Client satisfaction with nursing care SF-36 subscales: General health Physical function Role physical Role emotional Social function Bodily pain Vitality Mental health Nurse characteristics Age Experience with community nursing (years) Experience with agency (years) Employment status (full-time, part-time, casual) Hours worked per week Nurse Outcomes Work enjoyment Satisfaction with time for care Perception of job security Nurse retention Turnover rate Contract Characteristics Agency structure (for-profit or not-forprofit) Contract length Potential length of contract if renewed Volume of service Cost of nursing care Contract Structural Variables included the method for contracting and monitoring services, type of nursing services contracted, length of service contracted, the ownership type or structure (for-profit or not-for-profit), and the volume of nursing service awarded for each contract. The data on the ownership type for each provider agency that secured a 12

21 nursing service agreement and the volume of service were collated from the data collected in phase one of the study. If a new contract was subsequently issued, the data were updated. Information on the structures, policies, and resources to support community nurses was collected through a structured questionnaire that executive directors (or designates) of access centres and CEOs (or designates) of the nursing provider agencies were asked to complete. Quality of Care Variables. For the sample of study clients, data were collected on the number of registered nurse, registered practical nurse, and advanced practice/specialty nurse visits over the client s length of stay in the study, the continuity of care (hereafter referred to as consistency of nursing care provider), and the quality of technical and interpersonal care. The home health agency records were used to extract data on the number of visits provided over the length of stay in the study. Continuity of care was operationally defined as the proportion of visits made by the principal nurse assigned to the client s care. Client Outcome Variables focused on how clients and their health conditions were affected by the provision of home health nursing care. Two variables were included: health status and client satisfaction with nursing care. Health status data were collected at the time of the client s admission to the study and then again at discharge or after six weeks (for long-term clients) using the SF-36. The SF-36 has demonstrated reliability and sensitivity to nursing variables in a community setting (Irvine et al., 2000). The Client Satisfaction Scale developed by Reeder and Chen (1990) was used to collect clients perceptions of the quality of the technical and interpersonal aspects of care. The scale is a 35-item tool, assessing technical quality, communication, personal relationships between clients and providers, and general satisfaction. Reliability of the scale has been reported at.93 using Cronbach alpha. 13

22 Risk Adjustment. Risk adjustment (that is, the potential for good outcomes) was addressed with two approaches: 1) controlling for functional healthcare needs; and 2) controlling for age, medical diagnosis, and baseline scores on SF-36 subscales. To account for the clients baseline functional healthcare needs, the research team had planned to use the Resident Assessment Instrument-Home Care (Morris et al., 1997), which was being implemented at Ontario s access centres. This assessment instrument is a comprehensive standardized instrument for evaluating needs and strengths of homecare clients. It is designed to reference client complexity and provision of service issues for continuing care clients. Since its implementation was not province-wide at the time of the study and the data would not be available for the majority of study participants, it was decided instead to use the MI-Choice Screener. This research-based tool was originally designed as a telephone screening tool to determine if the more in-depth assessment with the resident assessment instrument was warranted. It was designed to identify people best served by information and referral services only, or a range of intensity of care options. Part A is a general high-level section, with items such as difficulty with housework, stamina/physical activity, bathing, skin problems, and cognitive skills. This variable is referred to as screener general in the presentation of results. Part B of the screener assesses specific instrumental activities of daily living, such as preparing meals, ordinary housework, managing medications, and shopping. This variable is referred to as screener IADL in this report. During the current study, the screener was completed by research staff during a telephone interview with study participants at the time of study enrolment. Cost Variables. The cost variables included 1) the centres direct costs associated with nursing service provision; and 2) the costs that clients assumed (either directly or through an insurance payer) for homecare services that were not provided under the publiclyfunded program. Data about costs incurred by the centres were abstracted from their databases. Costs incurred by clients were self-reported by through use of the Ambulatory and Home Care Record developed by Coyte and Guerriere (1998). Due to the low response rate (less than 25 percent) for client-incurred costs, these data have not been included in the analysis. 14

23 Nurse Outcomes. Data were collected on nurses professional job satisfaction, perception of job security, and retention. Professional job satisfaction refers to nurses perceptions of their job based on professional fulfillment and was assessed with the Nursing Job Satisfaction Scale (Atwood, Hinshaw, & Gerber, 1987). Its three subscales reflect the nurses perception of quality of care, enjoyment, and time to do one s job. The construct validity of the scale is supported, and its reliability and validity are documented, Cronbach s alpha Perception of job security was measured by a single item measured on a Likert scale (1-5). Staff turnover was measured by the number of nurses who left the organization as a proportion of the total number of staff employed in January of each year from 1997 to These data were provided by nursing agencies from administrative records. Participants Phase two of the study reflects nursing care that was managed by 11 community care access centres and provided by 11 different agencies. The characteristics of the participating nurses, clients, and the associated contracts are presented in this section of the report. Nurse Sample Of the 1,430 questionnaires distributed through the nursing agencies, 700 nurses returned a completed questionnaire for a response rate of 49 percent. There is anecdotal evidence that some questionnaires were undeliverable by the post office, and that some nurses were not actively working at the time of the study; thus the effective response rate is higher. Because participation was anonymous and recruitment was at arm s length by the agencies, the only follow-up was a thank you/reminder letter that was mailed to eligible nurses two weeks after the initial questionnaire. The demographic and employment characteristics of the nurses who responded to the questionnaire are summarized in the appendix, Table 22. The average nurse respondent was a 45-year-old female, working 29.1 hours/week, and employed in the community for 15

24 8.2 years. The average length of employment with the current agency was six years, but the high standard deviation indicated wide variation among respondents. Table 2: Nurse Employment Status, self-reported (n=697) Study Data Province of Ontario* (RN, all sectors) full-time 212 (30.4%) 54.0% part-time 266 (38.3%) 31.7% casual/relief not specified 219 (31.4%) 9.4% 4.9% * College of Nurses of Ontario, Membership Statistics 2002 Table 3: Nurse Employment Conditions mileage reimbursed by employer (n=695) 539 (77.0%) yes travel time reimbursed by employer (n=684) 291 (53%) yes benefits available (n=692) 381 (54.4%) no average hours per week with agency (n=687) mean 29.1 (sd 12.1) work for more than one employer (n=695) 208 (29.9%) yes Client Sample Data from 750 clients are included in the analysis. A total of 1,395 eligible clients were referred to the study by the centres. Table 23 in Appendix A documents the sample achievement. Table 4 describes the demographic data for clients who completed the demographic questions on the client questionnaire. Table 4: Client Demographics Characteristic Study Participants (# of respondents) gender (n=744) 455 (61.2%) female 289 (38.8%) male age (n=740) mean 63.7 years (sd 15.5; min 19, max 87) live with (n=738) 213 (28.9%) live alone 506 (68.5%) live with spouse or family member 19 (2.5%) live with other (e.g. friend, paid help, etc.) education (n=718) 403 (56.1%) 91 (12.6%) 149 (20.8%) 75 (10.5%) population subgroup 651 (89.9%) (n=724) 73 (10.1%) Comparability with all CCAC cases in Ontario, 2002 a evenly matched study population was approximately 5.7 years younger than the provincial mean more of the study participants lived alone, more lived with a spouse or family member, and fewer had other arrangements. completed high school or less on the job or formal technical training some college or university one or more university degrees Caucasian Other 16

25 Characteristic Comparability with all CCAC cases in Study Participants (# of respondents) Ontario, 2002 a marital status (n=737) 58 (7.9%) 447 (60.7%) 68 (9.2%) 164 (22.3%) single or engaged married or cohabitating divorced or separated widowed residence (n=738) 676 (91.6%) 44 (5.9%) 18 (2.5%) live in own residence (owned or rented) live in family member s home other new or ongoing client (n=750) 631 (84.2%) 119 (15.8%) newly admitted for nursing services long-term client, received care for >3 months a Source: Ontario Ministry of Health and Long-Term Care, CCAC Branch. Table 5: Anticipated Duration of Nursing Care at Time of Study Referral Anticipated duration of care # of study participants (%) 2 weeks or less 111 (14.8%) 3 weeks 80 (10.7%) 4 weeks 79 (10.5%) 5 weeks 28 (3.7%) 6 weeks or more 413 (55.1%) not specified 39 (5.2%) TOTAL 750 (100%) The most commonly represented diagnoses categorized by body system were: $ 21.4 percent (157) skin, subcutaneous tissue, and breast; $ 13.3 percent (98) cardiac or vascular; $ 12.2 percent (90) digestive system; and $ 10.6 percent (78) musculoskeletal. Carcinoma was recorded as the primary diagnosis for 153 clients (20.8 percent), represented proportionally among for-profit and not-for-profit agencies. A detailed description of diagnoses is in Appendix A, Figure 1. It is unknown if these are representative of all adult clients who receive nursing care through the access centres. Contract Characteristics Eleven different nursing agencies, which had a total of 34 service contracts with the participating access centres, are represented in phase two of the study. The study excluded overflow and single-client non-request for proposals contracts that are commonly used at some sites for challenging-to-serve clients and during peak demand 17

26 periods. Eighteen (52.9 percent) of the contracts were held by for-profit agencies. Ten of the 11 centres had awarded concurrent contracts to both for-profit and not-for-profit agencies. Contract characteristics are presented in Table 6. Nine access centres included palliative care within comprehensive nursing contracts, often highlighting specialty standards within the request for proposals. Two centres issued separate contracts for palliative nursing care, coincidentally to agencies that also held contracts for other types of nursing care. Since the nurse outcomes could not be differentiated (the same nurses provided care under both the palliative and general contracts) and the client sample for each palliative contract was too small (less than eight cases per contract) to be statistically valid, the palliative contracts are not identified separately in the study results. Table 6: Contract Characteristics Characteristic # of contacts per CCAC, excludes paediatrics and single-client contracts Phase 2: 11 CCACs mean 3.1 contracts (min 2, max 5) contract volume, visits + shift hours (n=31) mean 56,352 sd 27,760 Comparability with CCACs in Ontario* length of contract (months) mean 34.6, sd 7.4 mean 32.5, sd 11.2 potential length of contract if renewed (months) mean 52.2, sd 14.6 mean 49.04, sd n = 34 contracts a It is not feasible to compare the volume and number of contracts per CCAC because Phase 1 included all contracts, including single-client contracts and overflow contracts that are used at some CCACs to manage peak demand periods. Phase 2 did not include these low-volume contracts. a a Twenty-three of the 34 contracts in phase two included the potential for either one or two extensions, ranging from 12 months (n=4) to 60 months (n=2). Most (15 of 23) potential extensions were for 24 months each. Study Results Phase Two Approach to Data Analysis, Phase Two To examine the relationships among nurses and clients at the individual, contract, and provider agency levels, an advanced statistical methodology that takes into account these 18

27 different levels of hierarchy is warranted. If this hierarchy is ignored, then study results may lead to incorrect inferences. Hierarchical linear modeling techniques were used to examine the relationships between process variables and structural variables. Subsequent models investigated the relationship between nurse/client outcomes and process variables, while controlling for structural variables. To help select variables for the models, the stepwise regression approach to model building was conducted for client data. Variables were included in the model if p<0.15. Other variables of particular interest were also included in the model, whether or not they were selected from the stepwise procedure. In the models involving nurse outcomes, hours worked per week and work status were highly correlated. Therefore the hierarchical linear models were tested with the work status variables, excluding hours worked. Unless otherwise indicated, higher scores correspond to better nurse and client outcomes. Objective 1: Examine the relationship between contract structural variables and quality of care variables The quality of care was assessed with four variables: consistency of care provider, which was defined as the proportion of visits made by the principal nurse assigned to the client s care; the perceived quality of care from the client s perspective; and perceived quality of care from the nurse s perspective. Because of the recent literature suggesting better outcomes with a higher skill mix (Aiken et al., 2002; 2003; McGillis Hall et al., 2003; Needleman et al., 2002; Tourangeau et al., 2002), percentage of visits made by a registered nurse was also included as a variable. The descriptive results for the quality of care variables are summarized in Table 7, followed by a more detailed description of each variable. For study participants, a mean of 72.3 percent of visits were made by registered nurses (sd 38.68). 19