Patient Care Quality Program Final Evaluation Report

Transcription

1 Patient Care Quality Program Final Evaluation Report Prepared for: Prepared by: British Columbia Ministry of Health James Murtagh and Associates Date: July 2012

2 Acknowledgements This evaluation report was developed by James Murtagh and Associates (James Murtagh, Vicki Foerster, Paul Chaulk and Jennifer Jones) with input from the Ministry of Health. An Evaluation Steering Committee provided advice on evaluation design and the development of the evaluation plan. Its members included: Teri Collins - Ministry of Health Thomas Guerrero - Patient Care Quality Review Board Secretariat Mary MacKillop Provincial Health Services Authority Laura Nielsen - Vancouver Island Health Authority Claire Nishimura - Ministry of Health Val Stevens - Ministry of Health ii

3 Abbreviations & Acronyms used in this Report BC FOIPPA FTE HA HIPSC MOH NAO NHS NZ PCQ PCQO PCQRB PCQRBA PES PHSA PIA PSLS P/T QI SET UK USA British Columbia Freedom of Information and Protection of Privacy Act Full time equivalent Health Authority Health Information Privacy and Security Council Ministry of Health National Audit Office (of the NHS in the UK) National Health Service (United Kingdom) New Zealand Patient Care Quality Patient Care Quality Office Patient Care Quality Review Board Patient Care Quality Review Board Act Patient Experience Surveys Provincial Health Services Authority Privacy Impact Assessment Patient Safety and Learning System Province / Territory Quality Improvement Senior Executive Team (of health authorities) United Kingdom United States of America iii

4 Executive Summary British Columbia Health System The Ministry of Health (MOH) has overall responsibility for ensuring that quality, appropriate, cost effective and timely health services are available for all British Columbians. The MOH works with health authorities (HAs), health care providers, agencies and other organizations to guide and enhance the Province s health services to ensure that British Columbians are supported in their efforts to maintain and improve their health. The Province s six HAs are the organizations primarily responsible for health service delivery. Background Prior to 2008, each HA in BC had its own unique client relations function, with differing degrees of robustness. The complaints process was not always clear to the patient, and the outcome not always communicated. In addition, there was no clear mechanism for patients to escalate their complaints beyond the HA level. Patient complaints were not being tracked or reported on a provincial basis and there was no provincially coordinated means for identifying opportunities for quality improvement (QI) and sharing lessons learned. In 2008, the provincial government introduced the Patient Care Quality Review Board Act (PCQRBA) to establish a clear, consistent, timely and transparent process throughout BC for individuals to raise concerns regarding quality of care. Overview of the Patient Care Quality Review Board Act The Patient Care Quality Review Board Act (PCQRBA) came into force on October 15, On October 21, 2008 the Minister of Health publicly announced the launch of this new patient complaints process, including the appointment of the first PCQRB members. The Act required each HA to create a central Patient Care Quality Office (PCQO) to receive and respond to patient complaints. It also established six independent PCQRBs to review complaints not resolved at the HA level. The PCQRBs are independent from the HA and are accountable to the Minister. After completing a review, the PCQRBs may make recommendations to the HA and/or the Minister to improve the quality of patient care or the complaints process itself. Program Reach and Scope It is intended that all individuals who receive or expect but do not receive health care services under the jurisdiction of HAs may make a care quality complaint (as defined by the PCQRBA). Families, friends, advocates, or legal representatives may also complain on behalf of patients, clients, or residents, provided they have the written consent of the patient. Broadly speaking, care quality complaints include any health care services under HA jurisdiction with the exception of child care, involuntary admissions under the Mental Health Act, and decisions of the iv

5 Mental Health Review Board. Care quality complaints can relate to health care services that are delivered, funded, or contracted by HAs and include licensed care facilities (even if they are not HAfunded). Evaluation Rationale and Purpose The MOH commissioned a comprehensive Program Evaluation of the provincial Patient Care Quality (PCQ) Program. It was deemed to be an appropriate time in the development of the program to assess its implementation and administration. The evaluation is further supported by the MOH s recent commitments in Improving Care for B.C. Seniors: An Action Plan, released in February Finally, the MOH s commitment to undertake this evaluation appears to be well justified based on the evolutionary experience of other jurisdictions and the challenges still evident in the published literature. The Program Evaluation was intended to assess the implementation and administration of the Patient Care Quality Review Board Act, including its impact on various stakeholder groups (e.g., the transparency and accessibility of the complaints process for clients) in order to inform decisions for further program development and identify areas of risk and opportunities for improvement. It also assessed outcomes from the perspective of program clients and stakeholders. Evaluation Methods A program logic model and an evaluation framework were developed to guide the evaluation. A number of data sources were used to address the evaluation questions contained in the framework including document review, administrative data, stakeholder interviews, a PCQ client survey, and a literature review and other jurisdictional scan. There were a number of limitations to this evaluation, primarily related to the exclusions to the survey mailing list and to the survey response, which was mainly from more educated, English-speaking clients and permitted analysis only at a provincial level with limited breakdowns by demographic categories. Literature Review Conclusions Complaint management systems have long existed at the point of service in health care organizations. Escalation processes have also existed. More formal, statute-based systems / processes are a more recent phenomenon existing in some jurisdictions since the 1990s but still absent in others, including some, if not most, Canadian jurisdictions. Systems for managing complaints vary in form but all appear to be multi-staged in nature, with initial emphasis on local, relatively informal, resolution. There is some evidence to suggest these local options have been effective in most cases. Local point-of-service options are typically supplemented by one or two more formal options with the final option being an independent or quasi-independent review of the complaint, although the latter may be very limited with respect to types of cases examined. v

6 The published literature regarding complaints management tends to be implicitly or explicitly positive but this review found little in the way of empirical research regarding best practice or the overall impact of formal statute-based initiatives in the health sector. To the extent that evaluative literature exists, it appears to be predominantly from Australia, NZ and the UK. There is evidence to suggest complaint systems capture a minority of complaints and that the most formal structures serve an even smaller subset of the population. Awareness and socio-demographic factors are identified barriers to the use of complaint management systems. Challenges linking complaints to QI processes are also evident, as is fragmentation in the complaint process, i.e., the need for patients to deal with more than one process. These challenges persist to varying degrees, even in fourth generation complaint management systems. Evaluation Conclusions Although health care organizations have long managed complaints, their processes have frequently been variable and perhaps ad hoc in nature. Dissatisfaction with health services has historically been under-reported and avenues for appeal have been limited. The PCQ program is a statute-based complaints management program. It imposes a statutory obligation to ensure arrangements are in place for handling complaints while also ensuring a degree of consistency across the province. It is reflective of the approach taken in jurisdictions with the longest and richest documentation concerning complaints management in the health sector and is designed to facilitate system level insight. To the degree that statute-based systems are uniquely able to create system-level responses that reduce variability in complaint handling structures and processes, the PCQ program could be seen as replicating leading practice, at least at the macro level. Health sector stakeholders generally see the PCQ program as achieving its primary objectives. PCQ/PCQRB clients also display satisfaction levels equal to or better than available benchmarks. Improvements are desirable on several fronts but program promotion, access by minority groups and effective communication of outcomes all stand out as important areas for attention. Whether current program performance is adequate and whether the program provides value-for-money simply cannot be discerned at this point in time and with the available data. Recommendations Health sector stakeholders provided many suggestions for program improvement that are detailed in this evaluation report and these should be reviewed and considered further to the extent the following recommendations do not address them. There is little in the way of empirical or evaluative literature concerning complaint management in health care. In the absence of baseline and/or pre-defined targets specific to the PCQ program, the lack of empirical literature raises interesting challenges in terms of benchmarks for measuring performance or identifying best practice. This obviously has implications for the development of recommendations. vi

7 More specifically, without a clearly superior model, recommendations may tend to be less prescriptive and more oriented to exploration of options. Program Intent Recommendation 1: The MOH should review the intent of the PCQ program and be more explicit in the expression of that intent either by way of a statement of purpose or by way of a revised set of objectives. Recommendation 2: The MOH should review the complaint management streams under its control and provide clear guidance regarding how those streams interface, if at all. Data Recommendation 3: A review of complaint intake practices and the impact on complaint disposition and data capture should be undertaken. Recommendation 4: Options to gather information regarding latent complaints and complaints managed by processes other than the PCQ should be explored with explicit consideration given to adding questions to the patient experience surveys. Ongoing Evaluation Recommendation 5: Consideration should be given to exploiting the capacity inherent in the PSLS to capture real time satisfaction information from clients. Quality Improvement Recommendation 6: Consideration should be given to exploiting the capacity inherent in the PSLS to rank complaints by severity. Recommendation 7: Consideration should be given to better differentiating quality complaints from service complaints. Recommendation 8: The focus of the PCQ program should be on the core functions of complaint management including complaint capture and processing and caution should be exercised in having the PCQ program adopt an active QI function beyond that inherent in the effort to resolve specific complaints. vii

8 Program Promotion Recommendation 9: Existing program promotion strategies should be reviewed and a comprehensive communication plan developed. The plan should consider methods to ensure staff are familiar with the program and more direct marketing to clients such as provision of information at admission in the case of acute facilities. Attention should also be paid to ensuring program reach extends beyond acute care facilities. Resources Recommendation 10: The MOH should document existing PCQ office resourcing relative to workload and, in conjunction with the HAs, assess the implications for service delivery. To the extent that adverse implications are identified, the partners should identify potential solutions. Recommendation 11: Consideration should be given to allowing the PCQ officers the latitude to refer appropriate complaints to the point-of-care for expedited handling. Timelines Recommendation 12: Consideration should be given to establishing time limits for the filing of initial complaints and, more importantly, for the filing of independent reviews. Communicating Outcomes Recommendation 13: Current approaches to communicating actions flowing from the complaint process should be reviewed with input from a communications specialist. Recommendation 14: Consideration should be given to having PCQRBs review draft recommendations with HAs in advance of issuing a response to the client. The PCQRBs should not be bound to seek agreement regarding recommendations but the practicality of recommendations should be thoroughly discussed. To the extent this discussion results in an agreed course of action, the actions should be clearly communicated to the client. viii

9 Table of Contents 1. INTRODUCTION... 1 BRITISH COLUMBIA HEALTH SYSTEM... 1 BACKGROUND... 1 EVALUATION RATIONALE AND PURPOSE PROGRAM DESCRIPTION... 3 OVERVIEW OF THE PATIENT CARE QUALITY REVIEW BOARD ACT... 3 PROGRAM OVERVIEW... 3 PROGRAM GOVERNANCE... 4 PROGRAM REACH AND SCOPE... 5 STAKEHOLDERS... 5 RESOURCES... 5 REPORTING REQUIREMENTS... 6 PROGRAM LOGIC MODEL EVALUATION METHODS... 7 TYPE OF EVALUATION... 7 OVERALL APPROACH... 7 SCOPE OF EVALUATION... 7 EVALUATION FRAMEWORK... 7 DATA SOURCES... 7 EXISTING DATA SOURCES... 8 STAKEHOLDER INTERVIEWS... 8 CLIENT SURVEY... 9 DATA ANALYSIS LIMITATIONS LITERATURE REVIEW SUMMARY BACKGROUND ix

10 RESEARCH QUESTIONS METHODS RESULTS CONCLUSIONS FINDINGS ADMINISTRATIVE DATA STAKEHOLDER INTERVIEWS CLIENT SURVEY DISCUSSION CONCLUSIONS RECOMMENDATIONS APPENDIX A: PROGRAM LOGIC MODEL APPENDIX B. EVALUATION FRAMEWORK APPENDIX C. DATA COLLECTION INSTRUMENTS APPENDIX D. LIST OF INTERVIEWEES APPENDIX E. LITERATURE REVIEW REPORT x

11 1. Introduction British Columbia Health System The Ministry of Health (MOH) has overall responsibility for ensuring that quality, appropriate, cost effective and timely health services are available for all British Columbians. 1 The MOH works with health authorities (HAs), health care providers, agencies and other organizations to guide and enhance the Province s health services to ensure that British Columbians are supported in their efforts to maintain and improve their health. The MOH provides leadership, direction and support to health service delivery partners and sets province-wide goals, standards and expectations for health service delivery by HAs. The MOH enacts this leadership role through the development of policy, legislation and regulation, through funding decisions, negotiations and bargaining, and through its accountability framework for HAs and oversight of health professional regulatory bodies. The Province s six HAs are the organizations primarily responsible for health service delivery. Five regional HAs deliver a full continuum of health services to meet the needs of the population within their respective geographic regions. A sixth HA, the Provincial Health Services Authority (PHSA), is responsible for managing the quality, coordination and accessibility of services for province-wide health services and programs. These include the specialized programs and services that are provided through the following agencies: British Columbia (BC) Cancer Agency; BC Centre for Disease Control; BC Children s Hospital and Sunny Hill Health Centre for Children; BC Women s Hospital and Health Centre; BC Provincial Renal Agency; BC Transplant; Cardiac Services BC; the Emergency and Health Services Commission (which provides ambulance services across the province and operates BC Bedline, the provincial acute bed management system); BC Mental Health and Addiction Services and the Forensic Psychiatric Services Commission; and Perinatal Services BC. The MOH monitors the delivery of health services and the health of British Columbia s population on an ongoing basis. These monitoring activities inform the MOH s strategic planning and policy direction to ensure that the delivery of health services continues to meet the needs of British Columbians now and in the future. Background Prior to 2008, each HA in BC had its own unique client relations function, with differing degrees of robustness. The complaints process was not always clear to the patient, and the outcome not always communicated. In addition, there was no clear mechanism for patients to escalate their complaints beyond the HA level. Patient complaints were not being tracked or reported on a provincial basis and there was no provincially coordinated means for identifying opportunities for quality improvement (QI) and sharing lessons learned. 1 Adapted from Ministry of Health (2012). 2012/ /15 Service Plan. 1

12 In 2008, the provincial government introduced the Patient Care Quality Review Board Act (PCQRBA) to establish a clear, consistent, timely and transparent process throughout BC for individuals to raise concerns regarding quality of care. BC s actions are in line with other jurisdictions that have introduced statute-based complaint management processes, some as far back as three decades ago (e.g., Scotland). It is now widely accepted that patient complaints can be used to promote safety and reduce risk, provided systems exist to capture and systematically analyze complaints. Nonetheless, statute-based systems and other systems are not a panacea. Other jurisdictions have gone through multiple refinements in search of more effective approaches. Real challenges persist in capturing complaints, responding to complaints (organizational and individual) and capturing the potential quality benefit of a complaints system. Evaluation Rationale and Purpose The MOH commissioned a comprehensive Program Evaluation of the provincial Patient Care Quality (PCQ) Program. It was deemed to be an appropriate time in the development of the program to assess its implementation and administration. The evaluation is further supported by the MOH s recent commitments in Improving Care for B.C. Seniors: An Action Plan, released in February Finally, the MOH s commitment to undertake this evaluation appears to be well justified based on the evolutionary experience of other jurisdictions and the challenges still evident in the published literature. The Program Evaluation was intended to assess the implementation and administration of the Patient Care Quality Review Board Act, including its impact on various stakeholder groups (e.g., the transparency and accessibility of the complaints process for clients) in order to inform decisions for further program development and identify areas of risk and opportunities for improvement. This report outlines the evaluation approach and methods, findings from the each data source, a discussion of the findings, and conclusions and recommendations that flow from the evaluation. 2

13 2. Program Description Overview of the Patient Care Quality Review Board Act In the February 2008 Speech from the Throne, government committed to establishing a Patient Care Quality Review Board (PCQRB) in each health region. This was followed in May 2008 by the assent of the PCQRBA, which introduced a clear, consistent, timely and transparent patient complaints process across British Columbia. The Act required each HA to create a central Patient Care Quality Office (PCQO) to receive and respond to patient complaints. It also established six independent PCQRBs to review complaints not resolved at the HA level. The PCQRBs are independent from the HA and are accountable to the Minister. After completing a review, the PCQRBs may make recommendations to the HA and/or the Minister to improve the quality of patient care or the complaints process itself. The Act came into force on October 15, On October 21, 2008 the Minister of Health publicly announced the launch of this new patient complaints process, including the appointment of the first PCQRB members. Program Overview HA Patient Care Quality Offices As per best practices in complaints management, patients are encouraged to address their concerns at the time and place they occur with the person or unit that provided the service (i.e., at the point of care). If a patient remains dissatisfied, wishes assistance to have their issue resolved, or wants to make a formal care quality complaint, he or she can contact the health authority s PCQO. In accordance with the Legislation and Directives, the PCQO must acknowledge receipt of a care quality complaint within two business days. The PCQO then has 30 business days to investigate and a further 10 business days to respond to the complainant. This timeline may be extended with the complainant s consent, in which case the PCQO is required to communicate with the complainant every 20 business days. If a complainant is not satisfied with the response, or the PCQO has not responded within the mandated timeframe, the complainant may request a review by an independent PCQRB. Patient Care Quality Review Boards There are six independent PCQRBs, one designated for each HA. At a complainant s request, the PCQRB will review a care quality complaint that has first been addressed by the PCQO but remains unresolved. The Minister of Health may also direct a PCQRB to review any situation or matter. The Boards may make non-binding recommendations to the HAs and/or the Minister for improving the quality of patient care, and the quality of the complaints process itself. The PCQ Review Boards may review any care quality complaint regarding services funded or provided by a HA, either directly or through a contracted agency. They may also review complaints regarding services expected but not delivered by an HA (e.g., a complaint regarding a cancelled surgery). The 3

14 PCQRBs may only review complaints that have first been addressed by a HA s PCQO, unless otherwise directed by the Minister. If the PCQRB receives a complaint that cannot be reviewed, the complainant is redirected to the most appropriate body for his/her concerns. The PCQRB is expected to complete its review within a maximum of 120 business days and must report the outcome of individual case reviews, including any recommendations, to the complainant, the HA Board and the Minister. When a PCQRB makes a recommendation, the HA has 30 business days to respond, indicating what action will be taken. The first PCQRB members were appointed by the Minister on October 15, Each PCQRB has four to seven members chosen based on their expertise and experience. PCQRB members cannot include current HA employees, Board members or contractors. Collectively, the membership of each PCQRB should reflect: clinical experience; experience in procedural/administrative law and/or experience in adjudication or decision-making; knowledge of the health care system with an interest in QI; previous experience in client relations or a strong service orientation; and knowledge of, and an appreciation for, a broad range of cultural interests. Secretariat The six PCQRBs are supported by a shared Secretariat based in Victoria. The Secretariat works on behalf of the PCQRBs to receive and review requests, gather information from the PCQOs, prepare complaint case reports for the PCQRB to review, draft letters to HA and complainants, and prepare annual reports for the public. The Secretariat is the primary liaison between the PCQRBs, the PCQOs and the MOH. Ministry of Health The MOH s PCQ Program area provides strategic oversight to the provincial PCQ Program, and leads the development of policy, guidelines, regulations and directives to ensure program consistency and alignment with government direction. The PCQ Program area also works with HAs and other MOH program areas to implement QI initiatives in response to PCQRB recommendations. Patient Care Quality Working Group The MOH leads the provincial Patient Care Quality Working Group, with representation from each HA plus the Secretariat. The group meets bi-monthly to share information and address emerging issues. Program Governance Overall Program The MOH provides the provincial program with strategic direction, monitors compliance with legislation, and leads the development of policy, guidelines, regulations and directives as necessary. The MOH also monitors HA progress on implementation of actions related to PCQRB recommendations, and seeks opportunities for provincial QI where appropriate. The Program is accountable to the Assistant Deputy Minister of the HA Division. 4

15 PCQ Review Boards and Secretariat The MOH provides operational direction for the PCQRB Secretariat; however the PCQRBs are accountable directly to the Minister of Health. PCQRBs report annually to the Minister and the public on recommendations that have been made to HAs and proposed responses by HAs. PCQOs PCQOs are accountable to their respective HAs and report through their own governance structures. Each HA has a governing Board, Chief Executive Officer, and accompanying management structure. HA Boards are accountable to the Minister of Health. Program Reach and Scope It is intended that all individuals who receive or expect but do not receive health care services under the jurisdiction of HAs may make a care quality complaint (as defined by the PCQRBA). Families, friends, advocates, or legal representatives may also complain on behalf of patients, clients, or residents, provided they have the written consent of the patient. Broadly speaking, care quality complaints include any health care services under HA jurisdiction with the exception of child care, involuntary admissions under the Mental Health Act, and decisions of the Mental Health Review Board. Care quality complaints can relate to health care services that are delivered, funded, or contracted by HAs and include licensed care facilities (even if they are not HAfunded). Stakeholders The program s key stakeholders include: The general public who may need to use services of the BC health system Patients, residents and clients, or their families/representatives who have a care quality complaint as defined in PCQRBA HA boards, management and staff MOH management and staff Minister of Health PCQRB and Secretariat members Resources The provincial PCQ Program is supported by significant in-kind resources from the MOH. The MOH provides staff and resources to the PCQRB Secretariat. The PCQRB s ongoing annual budget allocation is in the range of $1.6 million. In 2009/10, the PCQRBs reported a total expenditure of $653,462, an amount which remained virtually unchanged in 2010/11. 5

16 The cost of establishing the PCQO in each HA in 2008 was covered by existing HA budgets. A PCQO s human resources may include a manager or director, Patient Care Quality officer(s) and/or administrative support. Reporting requirements The PCQRBs must report quarterly and annually to the Minister. June 30, 2010 marked the release of the PCQRBs first annual report (for fiscal 2009/10). The second annual report (for 2010/11) was released in June 2011 and the third annual report (for 2011/12) was released in July 2012 (see PCQ Program website: The PCQOs must report quarterly to their respective PCQRB and to the Minister. In addition, PCQOs report monthly to the PCQ Program area. Program Logic Model A program logic model outlining the inputs, activities, target groups, outputs, and intended outcomes of the program is presented in Appendix A. The program is broken down into three separate components (groups of related activities) including policy, oversight and monitoring; awareness; and complaint response. 6

17 3. Evaluation Methods Type of evaluation This evaluation was primarily formative (aimed at assessing how the program is implemented and administered and how it can be improved or further developed) with some summative elements (assessing outcomes from the perspective of program clients and stakeholders). Overall Approach A participatory approach was taken to the evaluation to ensure the evaluation approach was consistent with the planning and decision-making needs of the MOH and HAs and also to ensure that the evaluation findings are relevant and useful for ongoing program improvement. The evaluation consultants worked with an Evaluation Steering Committee comprised of representatives from the MOH, the Secretariat and the PCQOs. This committee provided feedback on the program logic model, evaluation framework, and evaluation plan. It also supported and assisted in the conduct of the evaluation. Finally, the committee provided the context needed to understand the evaluation findings. In addition, other MOH or HA representatives were contacted for further information and advice as needed (e.g., privacy officials, on-line survey developers, etc.). Scope of Evaluation The evaluation focused on the implementation and administration of the PCQRBA, responses to complainants by PCQOs and PCQRBs, and overall program oversight and supports including the Secretariat and the MOH. The evaluation also examined the relationship between the Program and other elements of the health system. Evaluation Framework An evaluation framework that outlines evaluation questions, indicators, data sources and timing of data collection is included in Appendix B. Data Sources A number of data sources were used to address the evaluation questions contained in the framework above including secondary (existing) data routinely collected and reported by the program and primary (new) data collected specifically for this evaluation. The data sources included: Document review Administrative data Stakeholder interviews (see Appendix C for a copy of the interview guide) Client survey (see Appendix C for a copy of the survey instrument) Literature review 7

18 Existing data sources Existing data sources used for this evaluation included document review, administrative data, and a broader literature review. The legislation/document/existing data review provided context, statistics on use of the various patient quality care processes prescribed in the Act, issues encountered, etc. Document review: Document review included a review of legislation, Ministerial Directives, and various program documents such as PCQRB Annual Reports, process flow and core functions charts, support tools such as the Best Practices in Complaints Investigations, Guidelines for Empathetic Response Letters, etc. Administrative Data: Administrative data included data submitted as per the MOH s minimum reporting requirements (as summarized in PCQRB annual reports), provincial complaints data trends reports, PCQ Working Group dashboard, and any other data provided by the MOH. Literature Review of health care complaints programs in other jurisdictions: The literature review and jurisdictional scan provided background and context to the evaluation as well as information on strategic directions and issues encountered in other jurisdictions. In addition to a literature search through formal databases, a general Internet search, and a specific review of websites of health care complaints programs in other jurisdictions, some jurisdictions were contacted via /telephone to ascertain whether additional information was available that is not posted on the program s website. Stakeholder Interviews A broad cross-section of stakeholders was interviewed to provide feedback on the process of implementing and administering the program as well as challenges faced, factors supporting program implementation and administration, and potential improvements for the future. Stakeholders interviewed included: MOH personnel involved with the program Secretariat director/review board officers PCQRB chairs and board members PCQO managers/directors The HA management staff members who interact most directly with the PCQOs A semi-structured interview guide was developed to guide the interviews and to ensure that key evaluation questions were addressed in each interview. Each interview also explored, in greater detail, issues particular to each interviewee. See Appendix D for the list of stakeholders interviewed. 8

19 Client Survey Purpose The client survey was conducted to inform key areas of the evaluation including how the client became aware of the program, client satisfaction with both the process and the resolution of the complaint, and areas for program improvement. Sampling Frame The sampling frame (eligible persons to be surveyed) included the clients who made a complaint to the PCQO according to a list that is maintained by the PCQO in each HA, as well as the list of clients escalating a complaint to the PCQRBs. The latter list is maintained by the Secretariat. (Note that only those clients for whom there was a complete Canadian postal address were included). The PCQO lists included all complaints placed with each PCQO, regardless of whether they were eventually brought forward to a PCQRB. The sample frame was the list of clients not a list of specific complaints (i.e., clients may have had multiple complaints but were only listed once). Two discrete sample frames were used: PCQO only: Clients who made a complaint to a PCQO only PCQRB: Clients who escalated their complaint to a PCQ Review Board The PCQO only sample frame was the list of clients who placed a complaint from October 1, 2010 to June 30, 2011 who did not have their complaint reviewed by a PCQRB. The rationale for restricting the sample frame to the time period after October 1, 2010 was to ensure that clients could recall the experience in sufficient detail. The rationale for restricting the sample to prior to June 30, 2011 was to only include complaints that have been resolved and communicated to the client, i.e., a number of more recent complaints may still be active. This time period included 4056 complaints a sufficient number from which to draw a survey sample. (Note that not all of these complainants were eligible depending on Canadian postal address availability and the exclusions described below.) The PCQRB sample frame was the list of clients who had a review accepted and completed with a PCQRB from April 1, 2010 to March 31, The start of the time frame pre-dated the time frame for PCQOonly clients as it takes longer to complete a review via the PCQRB system. There were approximately 137 total cases in this time period, some of which were still open complaints. A sample was selected to provide an adequate number of respondents to describe the PCQ program as a whole (at a provincial level). Sample Exclusions The Secretariat and each PCQO excluded any currently active cases from their respective samples. As noted above, complainants without a valid Canadian postal address were also excluded on the advice of privacy experts. Finally, if there were specific complaints in the consecutive sample that were 9

20 problematic to include in the survey sample, the Secretariat (for PCQRBs) and PCQOs removed these clients from their final sample. Criteria for exclusion included individuals: Currently involved in active litigation with the HA/PCQRB With a non-contact agreement in place with the HA/PCQRB With whom it is clinically contraindicated to communicate Known by Secretariat/PCQO staff to be deceased Identified at the time of complaint processing as a potential security risk Who have received a sensitive or highly private procedure, e.g., therapeutic abortion Privacy The Ministry completed one Privacy Impact Assessment (PIA) on behalf of all HAs prior to administering the survey. The PIA was shared with the Health Information Privacy and Security Council (HIPSC), which includes the director of privacy for each HA as well as MOH representation, for their information. The evaluation was a MOH-led initiative that HAs were required to participate in (as opposed to a partnership, which would have required HAs to approve the PIA). Regarding consent to contact patients: the Freedom of Information and Protection of Privacy Act (FOIPPA) stipulates that personal information can be used so long as the use of that information is consistent with the original reason for why it was collected. As such, each PCQO did not need to obtain consent to contact patients for the survey because, per the FOIPPA, the patient has already provided consent to use their contact information for anything related to their care complaint. The survey was presented as a QI project and a natural progression of the complaint process. The MOH removed all identifiers such as IP addresses, postal addresses, and names from the on-line survey datafile and hard copy surveys and released an anonymized survey datafile and anonymized hard copy surveys to the external evaluation consultants for entry and analysis. The consultants also had a contract requiring them to maintain confidentiality of all materials received as part of the evaluation. Instrument Development Survey development was guided by the evaluation framework to collect data on relevant evaluation questions and indicators. The survey included forced-choice (check box) questions with some openended questions to allow respondents to explain their responses or to provide other comments. A few key demographic questions were included for descriptive purposes and to assess how responses varied by demographic characteristics. The survey was available in English only and was reviewed for plain language by a literacy expert. The MOH prepared a letter inviting clients to participate in the survey process. The survey invitation letter included the option of calling an MOH contact number listed on the letter to ask questions. To the extent possible, the survey questions were designed to separate respondents perceptions of the process of handling their complaint (the main focus of the survey) and the resolution or outcome of their complaint. It was necessary to ask about the resolution of the complaint to assess how 10

21 respondents perceptions of the resolution of the complaint were associated with their responses on questions regarding the process of handling the complaint. Survey Administration The survey was administered via an invitation letter and hard copy paper surveys that were mailed to the selected sample by the PCQOs and Secretariat. A total of 806 surveys were mailed to valid postal addresses on, or shortly after, May 4 and responses were accepted until June 7, The hard copy surveys were returned by postal mail to the MOH. Potential respondents also had the option to go to a web address to complete the survey on-line using the approved MOH survey tool. Data Analysis Qualitative or textual information from stakeholder interviews and open-ended survey questions were analyzed according to standard qualitative criteria (Krueger, 1994) 2 in order to determine the major themes, for example: Language - the type of words that people use to express their views and/or experiences Context - the issues and/or situations that seem to stimulate a particular view or comment Consistency and/or diversity - whether the comments are generally consistent or diverse Specificity - connecting views and comments to specific individual experiences Frequency - how often a particular view or comment was expressed Intensity - how strong the particular point of view was made Numerical (quantitative) information from forced-choice survey questions was analyzed using descriptive statistics (e.g., proportions) and presented in table/chart format, where appropriate. In some cases, responses from two or more demographic groups were compared using inferential statistics (e.g., chi-squared tests) to determine if there were statistically significant differences between these groups. A more stringent criterion of p <0.01 was used to determine if differences were significant due to the large number of inferential tests performed. In addition to formal statistics, the results were examined for consistency of findings and whether an explanation for the findings could be derived. Numerical (quantitative) information from administrative data was reported as previously summarized in program documentation. 2 Krueger RA (1994). Focus Groups: A Practical Guide for Applied Research. Thousand Oaks, CA: Sage Publications. 11

22 Limitations There were a number of limitations to this evaluation: The survey sample frame was modified through the use of necessary exclusion criteria (as described above), which nevertheless opened the survey sample to potential selection bias. The response rate to the survey was less than anticipated (22%). The results from the survey respondents cannot be extrapolated to the full population of clients as it is not possible to determine the extent to which these clients represent the entire client population (i.e., demographic data are not collected by the program). Many survey respondents had post-secondary education and spoke English most often at home. Whether this is due to a bias in survey response or a true reflection of the composition of PCQ clients cannot be determined. There was limited sample size for some inferential statistics comparing various client survey subgroups such as comparisons by health authority or by the nature of the complaint. 12

23 4. Literature Review Summary The executive summary of the literature review is presented in this section. The full literature review report is included in Appendix E. Background The literature review of national and international information has been included in the evaluation of the PCQ Program to provide background and context. It also contributes information on evaluation approaches and indicators, strategic directions, and issues encountered. Research Questions How have jurisdictions beyond BC (nationally and internationally) structured their health care complaints systems? How have these systems been evaluated or reviewed, what indicators have been employed and what have the outcomes of evaluations or reviews been? Methods A number of publicly accessible bibliographic databases and internet resources were searched, limited to English language publications from 2006 to February Of the 945 references identified as potentially relevant, 51 were included in the review. Useful documents were also received from the BC MOH. For the Canadian jurisdictional review, knowledgeable people at the MOHs or HAs beyond BC were contacted by phone in March and April Results International Practices Published information for seven countries was located (Australia, Finland, Ireland, the Netherlands, New Zealand [NZ], the United Kingdom [UK] and the United States [USA]). Common themes and ideas emerged from the literature review of systems in other countries: Patients should understand how to provide complaints and compliments about their care, to both ensure optimal care for themselves and their families, and to help the system to improve. Access to complaints processes should be very clear and user-friendly. This may involve the assistance of a patient advocate to help a patient / family navigate the system. Early on, attention should be paid to acknowledging and clearly understanding the details and nature of a patient s complaint and motivation for complaining; this can lead to managing patients expectations of the system. Some complaints may not be appropriate and processes should be developed to deal with these at an early stage to prevent them from bogging the complaint system down. 13

24 To as great an extent as possible, complaints should be dealt with at a local level. If an informal local process does not satisfy the complaint, or is not feasible, there should be a clearly identified resource to approach. If this second stage process does not satisfy the complaint, or is not feasible, an independent process should be available to carry complaints to some type of resolution. It was acknowledged that some complaints may never be resolved, despite adequate processes. Comprehensive data should be collected for regular reporting purposes and also for trending and complaint systems should be linked to QI activities. Standards set by accreditation bodies such as Accreditation Canada and the USA Joint Commission are driving use of patient complaints data for QI. Canadian Jurisdictions In Canada s other 12 provinces and territories (P/Ts), there is variation based on size of the jurisdiction (population and geography), extent of regionalization, maturity of the complaints system, mandate of the provincial ombudsman, etc. All systems encourage local resolution of concerns and have subsequent processes, although these vary. Also, in all jurisdictions, complaints concerning competence of independent providers are managed by regulatory colleges. Several complaint system models appear across the country: Unique solutions: In the Northwest Territories and Nunavut, based in part on small populations Confined to a hospital: In Ontario (no regionalization) and the Yukon (only one hospital) HA-based: In Manitoba, Nova Scotia, Newfoundland and Labrador, and New Brunswick Single health corporation / HA makes it look like a provincial system: In PEI and Alberta Provincial system: In Quebec where the Ombudsman is the final arbiter and Saskatchewan with Provincial Quality of Care Coordinators at Saskatchewan Health who handle complaints not settled at the HA or Ombudsman level Interviews also explored topics like capture of complaints information in databases, use of a severity scoring system, reporting of results, and links between patient complaints and QI activities. The less centralized the system, the less likely is there to be a common data capture/analysis system or linkages to QI activities. With respect to evaluations, it was not clear that any Canadian jurisdictions have conducted formal evaluations of their patient complaints systems. Conclusions Complaint management systems have long existed at the point of service in health care organizations. Escalation processes have also existed. More formal, statute-based systems / processes are a more recent phenomenon existing in some jurisdictions since the 1990s but still absent in others, including some, if not most, Canadian jurisdictions. Systems for managing complaints vary in form but all appear to be multi-staged in nature, with initial emphasis on local, relatively informal, resolution. There is some evidence to suggest these local options 14

25 have been effective in most cases. Local point-of-service options are typically supplemented by one or two more formal options with the final option being an independent or quasi-independent review of the complaint, although the latter may be very limited with respect to types of cases examined. The published literature regarding complaints management tends to be implicitly or explicitly positive but this review found little in the way of empirical research regarding best practice or the overall impact of formal statute-based initiatives in the health sector. To the extent that evaluative literature exists, it appears to be predominantly from Australia, NZ and the UK. There is evidence to suggest complaint systems capture a minority of complaints and that the most formal structures serve an even smaller subset of the population. Awareness and socio-demographic factors are identified barriers to the use of complaint management systems. Challenges linking complaints to QI processes are also evident, as is fragmentation in the complaint process, i.e., the need for patients to deal with more than one process. These challenges persist to varying degrees, even in fourth generation complaint management systems. 15

26 5. Findings Administrative Data Administrative data already summarized by the program in various reports and dashboards were used to describe the number and type of complaints as well as trends over time. Figure 1 shows that the volume of care quality complaints received by PCQOs has remained relatively constant between 1200 and 1600 complaints per quarter (three months) while the volume of inquiries has increased over time. However, there may be definitional issues involved as some PCQOs may be more likely than others to classify a client contact as an inquiry. Figure 1: Volume of PCQO Client Contact over Time (April 2009 to September 2011). 1,800 1,600 1,400 1,200 1, Ext complaints Care quality complaints Inquiries The most common complaint subjects (nature of the complaint) related to the nature of the care received (e.g., deficiencies in care, inadequate assessment, medication) followed more distantly by attitude or conduct of professionals/staff; accessibility (e.g., wait time in emergency, wait time for surgery/procedures, delays in service); communication; and environment (e.g., housekeeping, parking, food services), as shown in Figure 2. 16

27 Figure 2: Most Common Provincial Complaint Subjects (April 2009 to September 2011) Care Attitude/conduct Accessibility Communication Environment The most common sector for which complaints were received was acute care, followed more distantly by emergency care, ambulatory care, residential care, and mental health and addictions (Figure 3). Figure 3: Most Common Sectors of Complaints (April 2009 to September 2011) Acute care Emergency care Ambulatory care Residential care Mental health and addictions 17

28 Figure 4 shows the volume of reviews completed by all PCQRBs over time (a subset of the PCQO complaints described above). Volume increased in 2009 but has remained relatively steady since then. Figure 4: Volume of PCQRB Reviews Completed Over Time (January 2009 to December 2011) Figure 5 outlines the reviews completed by PCQRBs in calendar years 2009, 2010, and The 177 reviews completed during this period resulted in 258 recommendations, most of which were directed to HAs with a few directed to the MOH. Between 62% and 92% of cases in each HA resulted in recommendations (data not shown). Between 62% and 88% of recommendations were accepted in each HA (data not shown). The most common type of recommendations (in descending order) were casespecific recommendations, staff education/reinforcement, new or revised regional process, communication, new or revised local process, and improved PCQO process (data not shown). Figure 5: Recommendations Made by PCQRBs (January 2009 to December 2011)

29 Stakeholder Interviews Semi-structured interviews were conducted with a variety of stakeholders representing the following organizations: Ministry of Health (n=11) Health Authorities (n=19) PCQRBs and Secretariat (n=8) Less formal interviews/discussions occurred with additional individuals (n=4). See Appendix D for a list of interviewees. The most comprehensive interviews involved HA PCQO staff and lasted 1-2 hours. Topics covered included: local PCQO organization/operation, legislation, program strengths and perceived client experience, perceived challenges and risks, and opportunities for improvement. Other interviews were shorter in duration (30-60 minutes) and focused primarily on program strengths, legislation, challenges, reporting and opportunities for improvement. PCQO Organization/Operation In most instances the HA PCQ offices fall within the broader HA quality portfolio. With one exception, the PCQO leads have no direct responsibility for QI. In several instances the PCQ offices are more closely aligned with risk management services and in one case the PCQ officers also serve as risk management officers. Reporting relationships for PCQO leads vary from director level to vice-president level and the majority of the leads have been involved in the program, in one way or another, since its inception. Staffing levels vary across the HAs and comparisons are difficult as some positions are part-time, others perform dual roles and, in one instance, PCQ-type resources are not consolidated under the auspices of the PCQ office. Information regarding case loads per officer was not universally available but it appears there may be as much as an eight-fold variation across the HAs. Even within HAs, balancing workload can be an issue as, with one exception, PCQ officers typically have either defined geographic or programmatic jurisdictions, both of which may impede distribution of workload. The professional backgrounds of PCQ officers vary with some employers favoring Master s-trained personnel and others not. Overall, professional backgrounds appear eclectic with a mix of clinical and non-clinical resources. A number of interviewees described a lack of opportunities for professional development. Salaries for PCQ officers also appear quite variable with a cited range from $60,000 or lower to $90,000. Risk management officers and QI consultants attract higher compensation; therefore, PCQ officers performing dual roles tend to earn considerably more than their single role colleagues. At least one PCQO lead believes salary levels are leading to retention issues. Program promotion is generally limited to posters, brochures and web sites. The posters and brochures were originally developed by the MOH and some PCQO leads believe the tone of the brochures is unnecessarily negative. In some, if not all, instances the PCQ offices have no ongoing role in the distribution of printed materials within HA facilities so the availability of such materials was unknown. 19

30 Just two PCQ offices have any ongoing role in the orientation of new HA staff. Some offices have developed process maps for HA staff reference but others appear to have no such documentation. Two PCQ offices have installed telephone menu systems to support complaint intake. Provided complainants are able to navigate the menu system they are linked directly to a PCQ officer. In all remaining PCQ offices, calls from complainants are received by an administrative assistant and triaged. The triage function can serve several purposes including responding to requests for information, redirection of the call to point of care or elsewhere and/or identification of and referral to the appropriate PCQ officer. Typically, details regarding complainants and complaints are entered into the Patient Safety & Learning System (PSLS) Complaints Module but there is a lack of clarity and/or variation associated with the triage process. In fact, interviewees noted the PSLS tends to be used in slightly different ways across the PCQ offices. Three PCQ offices indicated that all calls are entered into the PSLS but at least two other offices suggest only those calls that are triaged through to a PCQ officer likely make it to the PSLS. One PCQ office adds a second triage level involving PCQ intake officers but this process seems focused on separating less complex from more complex files. There are a number of non-mandatory fields in the PSLS and this is one source of variation. For example, one office routinely uses the severity scoring tool embedded in PSLS and another picks up the score from incident reports related to a specific complaint, if such exist, but otherwise severity scoring does not occur. Program reporting tends to be heavily oriented to volume, categories of complaints and trend data. Most PCQ offices report out to operations leaders, senior management and the HA board but the frequency of reporting to each group can vary from monthly to annually. In some instances, board reporting focuses strictly on PCQRB issues. Not all PCQ leads are included in presentations to senior management or the HA board. In general, interviewees are of the view that the program has evolved steadily. The early years of the program have focused on defining processes and building tools and these efforts are probably not complete. One PCQ office only went live on PSLS in May Relationships are generally viewed as positive but not always collegial. The MOH was seen by some as inclined to micro-manage and the relationship between the PCQ offices and the PCQRB are viewed as improving but sometimes strained. Role clarity was perceived as an issue in relation to the numerous other complaint management process which exist and is a major issue in the PCQ office/pcqrb dynamic. Legislation Although most interviewees had reviewed the PCQRBA at some point, some had not read the legislation in any detail or had not done so in quite some time. Comments regarding the legislation were limited and, to the degree comments were forthcoming, focussed more on the content of Ministerial Directives as opposed to the legislation per se. The most common comments regarding the legislation related to timelines. The timelines prescribed for responding to and managing complaint files were variously described as challenging or unreasonable. PCQ leads expressed the view that case volumes make the prescribed timelines generally challenging. 20

31 The greatest concern regarding timelines related to the time allowed to assemble PCQRB files and complex physician-related files. Clarity concerns were raised in regarding use of the term qualified privilege in the PCQRBA, guidance concerning expedited reviews, consent, definitions and the external complaint regulation. Identified legislative gaps included a statute of limitations for filing complaints or seeking reviews and the ability for PCQRBs to order ex gratia payments to complainants. It was also noted that the legislation generally requires complaints to be handled on a first-come, first-served basis and does not facilitate triaging according to case severity. Other issues attributed to legislation, perhaps incorrectly, related to the perceived need to respond to each incident identified by a complainant as opposed to being able to take a thematic approach. Some also saw the legislation as divisive in that it purportedly requires PCQRBs to comment on PCQ office performance. Program Strengths and Client Experience Commonly identified program strengths included the presence of a consistent and transparent provincial approach to complaint management including defined timelines and the presence of an independent review process. Also identified were the commitment of PCQ office and PCQRB members and staff. Other benefits included the establishment of client-relation-type functions in some HAs where none previously existed, enhanced accountability regarding complaint management, the opportunity for QI and the development of tools such as the PSLS. Some interviewees spoke of the potential to depoliticize complaint handling, to remove the handling of challenging complaints from front line staff, and the opportunity to understand what is important to patients/clients. It was also noted that the PCQRBs are resourced to bring a clinical lens to reviews an approach that might be absent in other complaint processes. Notwithstanding the foregoing, some voices also questioned whether the program offered a significant benefit over what previously existed. As for client experience, there was consistency in the perspective that more clients are happy with the process than with the outcome delivered by the process. As a result, several interviewees questioned the overall impact on patient/client satisfaction. Some noted that the PCQ offices are seen as agents of the HAs and unnecessary buffers to access to the PCQRBs. There was also some concern that the program formalizes complaints where the patient/client simply wanted to be heard. Finally, it was noted the program is predominantly serving English-speaking Caucasians. 21

32 Challenges and Risks Challenges and risks were identified by interviewees and summarizing these in a comprehensive manner poses some difficulties. Broad common themes included: Objectives: Greater clarity regarding program objectives would be desirable. Is the focus process-oriented (structure, timelines, etc.) or outcome-oriented (conflict resolution, QI, etc.) or, perhaps, both? At present, there is some tendency to see a process orientation as the default position. Scope: There was some concern the program might become a repository for a variety of issues, thus diluting its focus. The view was expressed that PCQ offices are fully occupied just managing issues emerging from health care transactions that have occurred or are in progress. Assigning PCQ offices a role in managing events that have yet to occur (e.g., drug shortages, professional service withdrawals, etc.) was seen by some (but not all) as problematic. Profile: Although workload continues to climb there was also a sense that program promotion has stalled or has plateaued. While the possibility of sparking significant increases in workload causes some anxiety there was also acknowledgement of the need for a well structured communication plan. Competing complaint avenues: The issue of competing complaint management avenues was a very common theme. In general, no solutions for the situation are proposed but the potential for confusion and conflict was noted. Also evident was the fact that even where the various avenues are under the control of the HA there may be no operational linkages. Rigidity: The program was widely seen by HA staff as being too rigid, particularly in regard to who must manage or own the complaint (i.e., inability to divert complaints to point-of-care staff where this is appropriate and acceptable to complainants) and the structure required around letters to complainants (i.e., the inability to group issues into themes). Resourcing: There were broad concerns regarding the adequacy of resources to support program delivery both at the PCQ office level and the PCQRB level. For the PCQ offices, adequate resources are a central issue to meeting prescribed timelines and, to a degree, complainant expectations. Data integrity: Variations in intake processes raised concerns regarding data integrity and/or comparability. Autonomy: There was concern both within PCQ offices and the PCQRBs regarding their ability to function in a reasonably autonomous manner. PCQ offices expressed some concern regarding linkages to, or oversight by, risk management and similar functions while some PCQRB members expressed concern about being seen as too closely linked to the HAs or the MOH. On the other hand, HA staff members desire more engagement with the PCQRBs and more discussion of potential recommendations. At the same time, PCQ offices also expressed concern regarding the management of files that clearly have claims potential. 22

33 Special populations: There was a perception that the program cannot serve residents of small communities or those in residential care effectively where personal relationships and fear of retribution are seen as significant barriers to any complaint process. Concerns also exist regarding the ability to effectively serve clients with mental health concerns. QI: The need for links to QI was widely identified but in practice such links are tenuous. There was also some concern that the program can disenfranchise point-of-care providers by moving complaint resolution away from point-of-care and thus eliminate the most obvious opportunity for system learning. PCQRB: There was notable concern within the PCQRBs regarding succession planning as well as a desire to permit PCQRB members to hear cases outside their home HA. On the latter point, PCQRB members were not advocating for the elimination of HA-specific boards but were advocating for less regional exclusivity in the structuring of specific review panels. This was seen as an avenue to member development, inter-regional learning and a means for managing workload. Opportunities for Improvement As with challenges and risks, interviewees identified a broad range of potential improvements, most of which require no explanation: Structural Work to increase the perceived or real autonomy of the PCQ offices and PCQRBs Consider rationalizing or in some way coordinating the competing complaint management avenues Operational Enhance program resourcing, particularly at HA PCQO level Clarify program definitions and the concept of qualified privilege Focus on high impact issues to reinforce credibility with clinicians Preclude consideration of complaints associated with financial claims Permit issues to be themed in complaint responses Provide a mechanism to address complaints seen to be frivolous or vexatious Permit complaints to be diverted to point-of-care, where appropriate and acceptable Modify promotional material to emphasize a QI orientation, including compliments and complaints Enhance linkages to QI Develop a data dictionary for the PSLS Timelines Introduce a statute-of-limitations for filing complaints and for requesting PCQRB review Review timelines for assembly of PCQRB packages and for complex case reviews 23

34 PCQRB Client Survey Limit PCQRB reviews to the information available during PCQO review of the complaint Require the PCQRB to consult regarding draft recommendations Permit PCQRB members to hear reviews in any HA Ensure adequate succession planning for PCQRBs Make provision for ordering compensation for complainants but clarify PCQRBs will not consider any matter related to legitimately levied fees Increase use of outside experts Response Rate and Sampling Accuracy (Margin of Error) Of 806 surveys sent to a valid postal address, 178 clients completed the survey (response rate 22%). This provided an overall survey accuracy (margin of error) of +/- 7.2%, 19 times out of 20, adjusted for the finite population of clients accessing the program during this time period (using conservative estimates of response proportions). Most surveys were completed in hard copy (69%) whereas the remainder were completed on-line (31%). The response rate was higher for PCQRB clients (37%) than for PCQO clients (21%). The PCQO response rates varied slightly by HA, ranging from 13% to 26%. The representation of respondents (the 178 clients who completed the survey) from the various HAs was: 3 Fraser HA PCQO - 24% Interior HA PCQO - 16% PHSA PCQO - 3% Vancouver Coastal HA PCQO - 17% Vancouver Island HA PCQO - 20% Northern Health HA PCQO - 1% PCQRBs (combined) - 20% Demographics of Survey Respondents Most survey respondents were the person who had made the original complaint (98%) with the remainder of respondents assisting someone to complete the survey. Whether the complaint was about the respondent s own care or someone else s care was roughly equal. Respondent s own care - 47% Someone else s care (e.g., making a complaint on behalf of a relative) - 49% Both - 4% 3 Totals may not add to 100% throughout this section due to rounding error. Missing survey data (non-response on individual survey items) are excluded throughout, unless otherwise noted. 24

35 Tables 1 to 3 outline the respondent demographics. Table 1 details the age of survey respondents compared to the broader population and shows that PCQ clients were more likely to be age 65 or older. Table 2 outlines the education level of survey respondents and shows that most respondents (79%) had post-secondary education. Table 3 shows that nearly all respondents spoke English at home. Table 1: Age of Respondents vs. Age of BC Population (2006 Census) Age % of survey respondents 2006 census data BC population 4 Under age 18 0% 21% Age 18 to 64 63% 65% Age 65 or older 37% 15% Table 2: Highest Level of Education Achieved Education % of respondents Less than high school 2% High school diploma 16% Community college or trades certificate or diploma 43% University degree 36% Other 3% Table 3: Language Spoken Most at Home Language % of respondents English 96% Other 1% Both English and other 3% 4 =0&PTYPE=88971,97154&S=0&SHOWALL=0&SUB=0&Temporal=2006&THEME=66&VID=0&VNAMEE=&VNAMEF= 25

36 History and Nature of Complaint In terms of complaint timing, 17% of respondents made the original complaint prior to or in Almost half made the complaint in 2011 or later while the remainder did not remember or indicate the timing: Up to % of respondents 2011 or later 43% Don t remember/missing - 40% Survey respondents were asked how they first heard about the PCQ Program (Table 4). The most frequent response was via a health professional, other professional, or hospital staff/administration (a combined 38% of respondents) followed by a website (14%) though it is not clear whether this is the PCQ website, an HA website, or other. Notably, some respondents made a complaint without being aware there was a formal program for dealing with complaints. Table 4: How did you first hear about the Patient Care Quality Program? Where Client First Heard About PCQ % of Respondents Health professional 28% Website 14% Pamphlet or poster 4% Family member or friend 4% Media 3% Other 37% Don t remember 10% 26

37 Additional methods noted by respondents in the other category were: Made a complaint but was not aware of the program - 11% of respondents Hospital staff or administration - 6% Not aware of the program prior to this survey - 5% Other professional (MLA, social worker, etc.) - 4% Research by self (Internet, called around, phone book) - 4% Respondent is employed in Health - 3% Health Authority/Ministry - 2% Letter from PCQ - 2% Other, uncategorized 1% Respondents were asked what service the complaint was about. The most frequent response was hospital inpatient, as selected by 41% of respondents (Table 5). Table 5: What service was your complaint about? Services % of Respondents* Hospital inpatient (hospital stays over 1 day, surgery) 41% Emergency room (emergency visit) 27% Same day hospital service (blood work, x-rays, cast change or removal, day surgery, minor procedures, IV therapy 17% Residential care (nursing homes, assisted living homes) 7% Home care (nursing care and/or support services received in your home) 4% Mental health or addictions 3% Ambulance 2% Other (most common was clinics (primary care/specialist/etc.) 8% Respondents were asked to indicate the nature of their complaint (Table 6). They were able to check more than one response, if applicable. Complaints were made about a variety of aspects of care, the most common being Attitude or behavior of health professionals or other staff as selected by 58% of all respondents. 27

38 Table 6: What was the nature of your complaint? Nature of Complaint % of Respondents* Attitude or behavior of health professionals or other staff 58% Medical care (i.e., medication, tests, diagnosis) 42% Getting services when needed (i.e., wait times for surgery, cancellations, getting into a residential care facility) 32% Condition of facilities (i.e., cleanliness, food service, parking) 24% Leaving the hospital (i.e., being discharged too soon or not soon enough) 18% How clear explanations or instructions were (i.e., home care instructions, explanations about medical conditions or procedures) 14% Other 30% *Respondents could select more than one response Common open-ended responses from the other category included: Poor medical care (physician/nursing direct care) Bed availability (sleeping in hallways, etc.) Communication Survey respondents selected one or more reasons for making the complaint (Table 7) with 75% of respondents indicating as the reason for making a complaint To prevent it from happening to other people. More than half of the respondents (61%) chose To have my complaint heard. Table 7: Why did you make the complaint? Reasons for Complaint % of Respondents* To prevent it from happening to other people 75% To have my complaint heard 61% To have my questions answered 43% To have actions taken against the staff or organization responsible 39% To receive an apology for what happened 29% To get more information about what happened 27% To get the service I was refused 24% Other 3% 28

39 Other common open-ended responses written in the other category included: To get financial compensation To change/improve care practices and/or policies Results of Making Complaint Awareness of the results of making the complaint was relatively low, less than 50% for each survey item listed in Table 8 below. Table 8: Summary of Awareness of Complaint Outcome Survey Item: % yes % no % not sure Were you made aware of what was done about your complaint? 48% 39% 13% Are you aware of any improvements that were made as a result of your complaint? Did you know that you can contact an independent Review Board if you are unhappy with how your complaint was dealt with? (PCQO only survey) 23% 66% 11% 29% 65% 6% There were no significant differences in awareness of what was done about the complaint or awareness of improvements made as a result of the complaint among demographic subgroups including PCQOs only versus PCQRBs (it is important to note that PCQRB clients had previously made their complaint with a PCQO); length of time since complaint was made; service the complaint was about; nature of the complaint; format of survey completion (paper versus on-line); whether the complaint was about the respondent s care or someone else s care; education or age group. Satisfaction with Complaint Table 9a presents detailed responses for the satisfaction items on the survey. The total percentage satisfied (very satisfied + satisfied + somewhat satisfied) is displayed in Table 9b. Satisfaction with the complaint process was relatively high for two items, How easy it was to make a complaint and How respectfully you were treated during the complaint process (81% of respondents were satisfied with each item). The lowest level of satisfaction was observed with How satisfied are you with what was done about your complaint (36% of respondents). Satisfaction with other items ranged from 50% to 68%, as summarized in Table 9b. 29

40 Table 9a: Detailed Breakdown of Satisfaction with Complaint Process and Outcome How satisfied were you with the following: Very Satisfied Satisfied Somewhat Satisfied Not At All Satisfied Don t know Information about how to make a complaint 20% 31% 17% 29% 4% How easy it was to make a complaint 20% 42% 19% 17% 2% Information about what happens after a complaint is made 12% 26% 23% 30% 9% How quickly your complaint was dealt with 12% 27% 21% 32% 8% How thoroughly your complaint was dealt with 10% 17% 23% 36% 14% How fairly your complaint was dealt with 13% 15% 22% 35% 16% How respectfully you were treated during the complaint process Overall, the way in which my complaint was dealt with How satisfied are you with what was done about your complaint 27% 37% 17% 17% 2% 13% 21% 22% 38% 7% 8% 14% 14% 45% 19% Table 9b: Summary of Satisfaction with Complaint Process and Outcome How Satisfied Were You With the Following: % Satisfied (Very Satisfied + Satisfied + Somewhat Satisfied) Information about how to make a complaint 68% How easy it was to make a complaint 81% Information about what happens after a complaint is made 61% How quickly your complaint was dealt with 60% How thoroughly your complaint was dealt with 50% How fairly your complaint was dealt with 50% How respectfully you were treated during the complaint process 81% Overall, the way in which my complaint was dealt with 56% How satisfied are you with what was done about your complaint 36% 30

41 There were no consistent, significant differences in satisfaction with aspects of the complaint process and the outcome among demographic subgroups including length of time since complaint was made, service the complaint was about, nature of the complaint, format of survey completion (paper versus on-line), whether the complaint was about the respondent s care or someone else s care, education, or age group. There were significant differences in satisfaction by whether the complaint was with a PCQO only versus being escalated to a PCQRB, awareness of what was done about the complaint, awareness of any improvements made as a result of the complaint, and satisfaction with what was done about the complaint (Table 10). Satisfaction with some aspects of the complaint process and outcome were influenced by whether follow-up on the complaint was by the PCQO only versus escalation to a PCQRB, as shown in Table 10. Satisfaction was higher among those who had their complaint dealt with by a PCQO only as opposed to those whose complaint was escalated to a PCQRB. (This is not surprising given that only those dissatisfied with the initial complaint response are subsequently escalated to a PCQRB.) Table 10: Satisfaction with Complaint Process and Outcome by PCQO only vs. PCQRB Clients. How satisfied were you with the following: Involved Organization PCQO only PCQRB Information about how to make a complaint NS NS How easy it was to make a complaint NS NS Information about what happens after a complaint is made* 76% 40% How quickly your complaint was dealt with NS NS How thoroughly your complaint was dealt with* 69% 29% How fairly your complaint was dealt with* 70% 33% How respectfully you were treated during the complaint process NS NS Overall, the way in which my complaint was dealt with* 70% 34% How satisfied are you with what was done about your complaint* 54% 13% * Statistically significant difference (p<0.01, chi-squared test) NS Not significant (p>= 0.01, chi-squared test) Satisfaction with many aspects of the complaint process and outcome were somewhat influenced by follow-up on the complaint by the PCQO and/or PCQRB (see Results of Making Complaint section above), as shown in Table 11. Satisfaction was higher among those who were made aware of what was done about their complaint for several aspects of the complaint process and, not surprisingly, for the 31

42 outcome. This effect was apparent even with satisfaction with elements of the program unrelated to how the complaint was dealt with, such as ease of making the complaint. Table 11: Satisfaction with Complaint Process and Outcome by Awareness of What was Done about the Complaint How Satisfied Were You With the Following: % Satisfied (Very Satisfied + Satisfied + Somewhat Satisfied) Aware of What Was Done About the Complaint? Yes No/Not Sure Information about how to make a complaint* 86% 54% How easy it was to make a complaint* 94% 73% Information about what happens after a complaint is made* 80% 53% How quickly your complaint was dealt with* 78% 53% How thoroughly your complaint was dealt with NS NS How fairly your complaint was dealt with NS NS How respectful you were treated during the complaint process NS NS Overall, the way in which my complaint was dealt with NS NS How satisfied are you with what was done about your complaint* 61% 29% * Statistically significant difference (p<0.01, chi-squared test) NS Not significant (p>= 0.01, chi-squared test) Satisfaction with most aspects of the complaint process and outcome were strongly influenced by whether the respondent was aware of improvements that were made as a result of the complaint (see Results of Making Complaint above), as shown in Table 12. Satisfaction was much higher for most aspects of the complaint process and, not surprisingly, the outcome among those who were aware of improvements made as a result of their complaint. This effect was again apparent even with satisfaction with elements of the program unrelated to how the complaint was dealt with, such as ease of making the complaint. 32

43 Table 12: Satisfaction with Complaint Process and Outcome by Awareness of any Improvements that were Made as a Result of the Complaint % Satisfied (Very Satisfied + Satisfied + Somewhat Satisfied) How Satisfied Were You With the Following: Aware of any Improvements that were Made as a Result of Your Complaint? Yes No/Not Sure Information about how to make a complaint NS NS How easy it was to make a complaint NS NS Information about what happens after a complaint is made* 92% 60% How quickly your complaint was dealt with* 97% 56% How thoroughly your complaint was dealt with* 97% 46% How fairly your complaint was dealt with* 92% 49% How respectful you were treated during the complaint process NS NS Overall, the way in which my complaint was dealt with* 92% 51% How satisfied are you with what was done about your complaint* 91% 27% * Statistically significant difference (p<0.01, chi-squared test) NS Not significant (p>= 0.01, chi-squared test) Satisfaction with all aspects of the complaint process was strongly influenced by whether the respondent was satisfied with the outcome of the complaint, as shown in Table 13. Satisfaction was much higher (above 90%) for all aspects of the complaint process if the respondent was satisfied with what was done about the complaint. This effect was again apparent even with satisfaction with elements of the program unrelated to how the complaint was dealt with, such as ease of making the complaint. 33

44 Table 13: Satisfaction with Complaint Process by Satisfaction with Outcome of the Complaint. % Satisfied (Very Satisfied + Satisfied + Somewhat Satisfied) How satisfied were you with the following: Satisfied with what was done about your complaint? Very satisfied / satisfied / somewhat satisfied Not at all satisfied Information about how to make a complaint* 90% 58% How easy it was to make a complaint* 95% 67% Information about what happens after a complaint is made* 98% 40% How quickly your complaint was dealt with* 93% 40% How thoroughly your complaint was dealt with* 98% 22% How fairly your complaint was dealt with* 98% 27% How respectful you were treated during the complaint process* 97% 67% Overall, the way in which my complaint was dealt with* 98% 25% How satisfied are you with what was done about your complaint N/A N/A * Statistically significant difference (p<0.01, chi-squared test) N/A Not applicable (same question) If survey respondents indicated they were not satisfied with the above survey items, they were asked in an open-ended question why they were not satisfied. Their responses included: No satisfactory result of making the complaint - nothing or very little was done as a result of the complaint (72 respondents) The complaint was not taken seriously; complainant was not treated fairly, was disrespected or blamed, or was not listened to; the letter of complaint was not fully read, facts were misstated, or there were errors in information (47) Process takes too long, either to get a response (months) or to get a resolution (years) (36 respondents) 34

45 No follow up or feedback to the complaint - either no response at all to the complaint (until they received the evaluation survey) or an initial phone call or form letter and no further communication (32) No information available about how to make a complaint; it was hard to find out how to make a complaint, or had to go through multiple people to make a complaint (22) The PCQRB has no authority to enforce anything; is not independent enough; or it is all political (16) Strengths In an open-ended question (i.e., writing responses in their own words), respondents were asked to list three things that they found helpful about the complaint process. The most common themes (expressed by five or more respondents) and areas of strength included: Program staff (respectful, positive, sympathetic, helpful, empathetic, etc.) 38 respondents Being heard/listened to; being able to vent 30 Timely/prompt response 26 That the process exists; the ability to make a complaint 21 Possibility of a resolution; explanation and steps to take to get there - 20 Easily accessible 18 Nothing was helpful 18 Felt someone cared/someone responded 14 Being informed/follow up 6 Areas for Improvement Respondents were also asked to list three things that could be done to improve the complaint process. The most common themes (expressed by five or more respondents) and areas where there are opportunities for improvement included: Provide information of the process and follow up during the process 30 respondents Increase awareness of and access to the program; more public visibility 23 Investigate complaints fully and accurately 22 Have trained program staff (listen, sympathy, empathy, knowledgeable, assistive, etc.) 21 Provide notification of resolution or actions taken in writing 21 Respond and act quickly 20 Ensure fair and effective resolutions that are enforced 15 Speed up the time it takes to complete the process 7 Make the Board an independent body and increase authority of Review Board to take action 6 35

46 6. Discussion It may be argued that benchmarks and traditional notions of value-for-money must be applied sparingly in assessing the PCQ or similar programs. Although the literature is universally positive regarding complaint management, it is also clear that most complaints about health services never find voice. In a 2008 report, the National Audit Office (NAO) in the UK estimated that 14% of National Health Service (NHS) clients were in some way dissatisfied with their experience, 5% of whom proceeded to make a formal complaint. 5 In other words, less than 1 in every 100 dissatisfied clients proceed to make a complaint. The chief reason for not complaining was lack of confidence that anything would change. Data from The Netherlands may indirectly reinforce the theme. Christiaans-Dingelhoff et al. reviewed 375 patient charts in 14 hospitals with a view to identifying adverse events and the degree to which those events are reported in complaint, medico-legal or incident report databases. 5 They identified 498 adverse events, only 18 of which were reported by patients or health care professionals. 6 While health care organizations have long had processes for managing complaints it is clear those processes, some of which were quite informal, have done little to encourage or facilitate complaints capture and handling. Further, it is clear that ad hoc approaches and even the more formal complaint avenues long present in some systems could not address the volume of complaints potentially lurking in the system. Against this backdrop and a broader recognition of the potential link between complaints and QI/patient safety, formal initiatives such as the PCQ program are necessary and may well need to be accepted as valuable regardless of yield in terms of complaint volume or outcome, at least in the medium term. The PCQ program can be characterized as a statute-based complaints management program. In this regard the PCQ program is more like complaint programs in the UK and Australia than those present in other Canadian jurisdictions. As noted by the NAO, modern organizations must be proactive in identifying and managing complaints. The PCQRBA imposes a statutory obligation to ensure arrangements are in place for handling complaints while also ensuring a degree of consistency across BC. To the degree that statute-based systems are uniquely able to create system level responses that reduce variability in complaint handling structures and processes, it is arguable the PCQ program is replicating leading practice, at least at the macro level. Based on documentation dating from the establishment of the program, the program s primary objectives (short-term) relate to clear, consistent, timely and transparent processes which provide clients with a thorough and respectful response to their complaint. Longer term objectives are similar but also include quality improvement. 5 National Audit Office. Feeding Back? Learning form Complaints Handling in Health and Social Care. London: The Stationary Office, October Available at: 6 Christiaans-Dingelhoff, I. et. al. To what extent are adverse events found in patient records reported by patients and healthcare professionals via complaints, claims and incident reports? BMC Health Services Research 2011, 11:49. 36

47 Health sector stakeholders generally perceive the PCQ program to be delivering on its short-term objectives. In identifying program strengths, stakeholders frequently referred to consistent, timely and transparent processes. The QI objective is approached more cautiously, especially when considering system level learning. Clients are more guarded or more discriminating in their assessment of program performance. A majority expressed satisfaction with information related to making a complaint (68% satisfied) and the ease of making a complaint (81%). Most were satisfied with how quickly complaints are dealt with (60%) and how respectfully they were treated (81%). However, only 50% were satisfied with how thoroughly their complaint was dealt with and just 36% were satisfied with what was done about their complaint. 7 Whether the foregoing figures suggest adequate program performance is an open question. The 36% satisfaction with what was done about complaints hardly seems encouraging but the result may not be as concerning as it first appears. Although benchmarks are hard to find, the UK NAO has published data regarding the performance of the NHS complaints system. It is not possible to map all dimensions of the PCQ client survey to the UK data but on the dimensions related to information about how to make a complaint and ease of making a complaint the PCQ results appear to be equal to or superior to those in the UK. Notwithstanding the 36% satisfaction with what was done about complaints (59% in the UK), overall satisfaction with complaint handling in the PCQ program is almost double that recorded in the UK (56% versus 27%). The superior UK performance on satisfaction with what was done with complaints may be attributable to communication regarding improvement flowing from the complaints (34% in the UK versus 23% in the PCQ survey). The literature suggests that evidence of change is of critical importance to complainants but that clients are frequently not informed or poorly informed. 8, 9, 10 The significance of this factor is clearly evident in the PCQ survey. As presented earlier in this report, clients who reported being aware of improvements flowing from their complaint expressed almost 100% satisfaction with most dimensions of the complaint process (see Table 12). The importance of communicating outcomes may be the most important observation flowing from this evaluation. While the foregoing discussion accepted 36% as a worst case proxy for PCQ performance and offered an explanation as to why that figure might be lower than the UK experience, the explanation may ultimately be irrelevant. The 36% figure is a blended figure combining PCQ and PCQRB clients. A more appropriate comparison to the UK data would entail looking at the PCQ clients in isolation from PCQRB clients as the UK data relates to primary complaint management and does not mix an appeal population. Satisfaction levels for PCQ clients are presented in Table 10. While some dimensions show no change in satisfaction others including how thoroughly complaints were dealt with, overall satisfaction with how complaints were dealt with and satisfaction with what was done rise notably (from 50%, 56% and 36% 7 The percentages presented in this paragraph are blended figures combining both the PCQ and PCQRB clients. 8 National Audit Office. 9 Friele, RD et al. Patient expectations of fair complaint handling in hospitals: empirical data. BMC Health Services Research 2006, 6: Friele, RD et al. Complaints handling in hospitals: an empirical study of discrepancies between patients expectations and their experiences. BMC Health Services Research 2008, 8:

48 to 69%, 70% and 54% respectively). At these levels the PCQ satisfaction levels are superior to the UK results on most measures and roughly equal (54% versus 59%) on the previously offending (36%) dimension of satisfaction with what was done about complaints. The general similarity of PCQ performance to that of other systems may get further reinforcement from Australian data, but it is less clear whether the comparison should be to PCQO or PCQRB results. Several Australian states regularly undertake and report the results of client surveys. 11, 12, 13 The survey populations range from and response rates range from 8% - 36% with the largest response generating 256 surveys. As with the BC and UK data, satisfaction with process elements, including overall satisfaction with how complaints were dealt with, tends to be relatively high (60%+) while satisfaction with what was done is lower at 45% - 56%. Separating the PCQ data from the PCQRB data highlights a potential performance differential that may be further accentuated if the Australian comparison should be to the PCQRB. Little comment can be offered on this point. The PCQRB sample size is too small to support further analysis. There may be a communication of results effect similar to what was seen with the PCQ clients but if so it appears to be amplified. It is arguable that the PCQRB has less capacity to make change and report back to clients especially to the degree it is limited to a review of review function. Perceived autonomy may also be at play as this is also a critical issue for complainants. 11 Health Quality and Complaints Commission. Annual Health Check 2011: A snapshot of our work in resolving healthcare complaints and monitoring the quality of health services. HQCC, Available at: %20final%20-%20high%20res%20-%20May% pdf 12 Health & Community Services Complaint Commissioner. Annual Report HCSCC, Available at: 13 Office of the Health Services Commissioner. Annual Report OHSC, Available at: pdf 38

49 7. Conclusions Although health care organizations have long managed complaints, their processes have frequently been variable and perhaps ad hoc in nature. Dissatisfaction with health services has historically been under-reported and avenues for appeal have been limited. The PCQ program is a statute-based complaints management program. It imposes a statutory obligation to ensure arrangements are in place for handling complaints while also ensuring a degree of consistency across the province. It is reflective of the approach taken in jurisdictions with the longest and richest documentation concerning complaints management in the health sector and is designed to facilitate system level insight. To the degree that statute-based systems are uniquely able to create system-level responses that reduce variability in complaint handling structures and processes, the PCQ program could be seen as replicating leading practice, at least at the macro level. Health sector stakeholders generally see the PCQ program as achieving its primary objectives. PCQ/PCQRB clients also display satisfaction levels equal to or better than available benchmarks. Improvements are desirable on several fronts but program promotion, access by minority groups and effective communication of outcomes all stand out as important areas for attention. Whether current program performance is adequate and whether the program provides value-for-money simply cannot be discerned at this point in time and with the available data. 39

50 8. Recommendations Health sector stakeholders provided many suggestions for program improvement and these should be reviewed and considered further to the extent the following recommendations do not address them. As previously noted, there is little in the way of empirical or evaluative literature concerning complaint management in health care. In the absence of baseline and/or pre-defined targets specific to the PCQ program, the lack of empirical literature raises interesting challenges in terms of benchmarks for measuring performance or identifying best practice. This obviously has implications for the development of recommendations. More specifically, without a clearly superior model, recommendations may tend to be less prescriptive and more oriented to exploration of options. Program Intent A relatively common theme in the stakeholder interviews related to clarity around the PCQ program intent. In responding to questions, stakeholders often indicated that their response would depend on whether the program was focussed on providing a process for managing complaints, the resolution of complaints or the identification of opportunities for improvement. This is an interesting issue. The program s primary objectives seem oriented to process with little explicit reference to resolution or improvement. On the other hand, clients clearly attach high value to resolution. Clarity around program intent is probably not aided by the existence of parallel or competing complaint management streams including some under the control of the MOH and/or HAs. While there may be justification for multiple streams it is unlikely that this makes navigation any simpler for clients. Stakeholders are also unclear and/or in open disagreement regarding how these various options link together. Recommendation 1: The MOH should review the intent of the PCQ program and be more explicit in the expression of that intent either by way of a statement of purpose or by way of a revised set of objectives. Recommendation 2: The MOH should review the complaint management streams under its control and provide clear guidance regarding how those streams interface, if at all. Data Complaint intake processes clearly vary across PCQ offices and there continues to be some doubt that PCQ offices are consistently capturing cases in the PSLS. No examination of this issue occurred in the course of this evaluation but the doubt expressed by some stakeholders seems reasonable. Much less clear is what impact the possible variation actually has on program integrity. The integrity issue is also interesting from the perspective that the PCQ program does not have exclusive jurisdiction over any particular category of complaints. It is a given that identical complaints may more often be handled on a less formal basis at the point-of-care or elsewhere via competing complaint management streams. These complaints are not captured in the PSLS. Any future assessment of the value for money offered by the PCQ program would require a better understanding of the complaint landscape. In particular, additional insight into the latent complaint pool is required as is the use or non-use of alternative resolution processes, outcomes, etc. 40

51 Recommendation 3: A review of complaint intake practices and the impact on complaint disposition and data capture should be undertaken. Recommendation 4: Options to gather information regarding latent complaints and complaints managed by processes other than the PCQ should be explored with explicit consideration given to adding questions to the patient experience surveys. Ongoing Evaluation The current evaluation was limited to a degree by the absence of baseline data and challenges associated with efforts to engage clients. The results of this evaluation provide a baseline of sorts for future assessment work and it would be desirable to have client satisfaction and possibly other measures collected on a more ongoing basis. The PSLS includes a non-mandatory data field related to client satisfaction which at present is generally not used. It seems reasonable to think that an inquiry regarding satisfaction would be the natural conclusion of a complaint management process. Recommendation 5: Consideration should be given to exploiting the capacity inherent in the PSLS to capture real time satisfaction information from clients. QI Many stakeholders recognize the potential contribution of complaint data to QI and there is some desire to see the focus of the PCQ program shift more towards utilization of complaint data for this purpose. However, current data practices may not easily support a QI focus, i.e., it does not appear that current data capture includes any severity ranking of complaints (with the exception of one PCQ office) or any effort to differentiate quality complaints from more service-focussed complaints. In essence every complaint is of equal consequence and all are assumed to involve care quality issues. The foregoing issues merit consideration but so does the question of the PCQ program role in QI. An underlying purpose of most complaint management programs is to capture some portion of the large volume of latent complaints in the health system. While it is understandable that complaints management programs might wish to engage in downstream activities such as QI it is important that such an orientation not be taken at the expense of core upstream program responsibilities (e.g., capturing and processing complaints). It is also important that the potential to focus on QI not be permitted to fragment QI efforts. A system level (or HA level) QI effort should seek information from a broad range of disparate sources (i.e., incident reports, coroner s reports, complaints, patient experience surveys, claims files etc). In this regard the QI function has a very important integrative role. Caution should be exercised in pursuing a QI role for the PCQ program. The PCQ program should certainly feed the QI system but as suggested - above, a PCQ-specific QI function could distract from core PCQ functions and fragment QI efforts. 41

52 Recommendation 6: Consideration should be given to exploiting the capacity inherent in the PSLS to rank complaints by severity. Recommendation 7: Consideration should be given to better differentiating quality complaints from service complaints. Recommendation 8: The focus of the PCQ program should be on the core functions of complaint management including complaint capture and processing and caution should be exercised in having the PCQ program adopt an active QI function beyond that inherent in the effort to resolve specific complaints. Program Promotion International experience suggests that awareness of and ability to navigate complaint programs is a major issue for patients/clients. Some jurisdictions have established advocacy programs to assist in navigating the complaints process but experience shows awareness of these programs is also problematic. The literature also shows that some demographic groups tend to underutilize programs. Promotion of the PCQ program is limited and it appears clients bear significant responsibility for finding the program. The PCQ client survey revealed that health professionals are the most common point of referral and yet it appears the PCQ program is absent from almost all staff orientation programs. (In contrast. more than 80% of NHS staff members are oriented to complaints management upon hire.) The PCQ client survey also indicated the program is being used primarily by relatively well educated Englishspeaking clients. Whether this is an artifact of the survey sample or reflective of the demographic issues identified in the literature is unknown. Numerous stakeholders noted the PCQ program lacks a comprehensive communication plan. Recommendation 9: Existing program promotion strategies should be reviewed and a comprehensive communication plan developed. The plan should consider methods to ensure staff are familiar with the program and more direct marketing to clients such as provision of information at admission in the case of acute facilities. Attention should also be paid to ensuring program reach extends beyond acute care facilities. Resources Resourcing was a common theme throughout the interview process. No workload audit was carried out so it is difficult to offer definitive guidance. As a reference point, the NHS employs 880 FTEs to manage roughly 133,000 complaints which equates to about 151 complaints/fte. It is not clear what types of staff members are included in the NHS figure or, more specifically, whether this includes support staff. The average cost to manage a complaint in the NHS is 640/case and the average time to close a file is 23 working days National Audit Office. 42

53 A noteworthy lesson from the UK is that, to the degree complaint programs surface latent complaints, the volume of work can grow quite significantly and strain the capacity for timely response. Given the QI value of local complaint resolution, the diversion of appropriate cases (i.e., not involving multiple departments/providers, acceptable to the complainant, likely resolvable in hours) to the point-ofcare may represent a partial alternative to increased resourcing for PCQ offices. Recommendation 10: The MOH should document existing PCQ office resourcing relative to workload and, in conjunction with the HAs, assess the implications for service delivery. To the extent that adverse implications are identified, the partners should identify potential solutions. Recommendation 11: Consideration should be given to allowing the PCQ officers the latitude to refer appropriate complaints to the point-of-care for expedited handling. Timelines Numerous comments were made concerning the timelines for managing complaints, e.g., issues included the timelines for filing complaints, assembling complex files, and filing requests for review. The adequacy of timelines could be debated for some time. Generally, experience suggests complaints should be managed as quickly as possible and from this perspective total processing time is more relevant than time allowed for any of the intermediate steps. A modest majority (60%) of PCQ clients indicated that they were satisfied with how quickly their complaints were managed. This would seem to mitigate against any extension of the overall timelines. While the intermediate timelines are more of an administrative issue, it is clear that any change would need to come at the expense of some other stage of the process. Some jurisdictions impose timelines concerning the filing of initial complaints or appeals. To the extent that these exist, they are not hard deadlines as exceptions can be made in certain cases. The value of these provisions is unclear. Recommendation 12: Consideration should be given to establishing time limits for the filing of initial complaints and, more importantly, for the filing of independent reviews. Communicating Outcomes It is absolutely clear that communicating outcomes is critical to client satisfaction with the complaint management process. The PCQ client survey suggested that many clients do not know what was done with their complaint and an even smaller percentage was aware of any improvements flowing from their complaint. It is unclear whether this reflects a failure to deliver the message or a problem receiving the message. Once again, experience overseas suggests that communication with clients can be of poor quality and place excess emphasis on policy and procedural detail. The importance of outcomes may also have interesting implications for PCQRBs. Unreasonable recommendations and failures to act on recommendations were common themes in stakeholder interviews. While it is important that PCQRBs not be co-opted by HAs, divergence in recommendations and actions will almost certainly translate to client dissatisfaction. Similarly, an inability on the part of the PCQRB to communicate change to its clients will likely adversely impact client satisfaction with the PCQRBs. Some Australian states require consultation before finalization of recommendations. 43

54 Recommendation 13: Current approaches to communicating actions flowing from the complaint process should be reviewed with input from a communications specialist. Recommendation 14: Consideration should be given to having PCQRBs review draft recommendations with HAs in advance of issuing a response to the client. The PCQRBs should not be bound to seek agreement regarding recommendations but the practicality of recommendations should be thoroughly discussed. To the extent this discussion results in an agreed course of action, the actions should be clearly communicated to the client. 44

55 Appendix A: Program Logic Model The program logic model is displayed on the following page. 45

56 46

57 * From Ministry of Health (2012). 2012/ /15 Service Plan. ** Health authorities include all services, programs and agencies for which they are responsible. 47

58 Appendix B. Evaluation Framework PCQ Evaluation Matrix Activities Evaluation Questions 1. How has the implementation and administration of the program evolved over time? 2. To what extent is the program administered similarly or differently across HAs and what impact, if any, does this have? 3. What is the client experience with various aspects of the program? 4. What are the strengths of the program? 5. To what extent is the legislation complete and appropriate for the current program context? Indicators Description of how the program has evolved over time with regards to: Policy, oversight and monitoring Awareness Complaint response Reasons for changes in program Description of common program elements Program elements which vary across HAs Consistency of reporting practices Perceived impact of specific ways in which program is organized Client experience with various aspects of the program Accessibility Timeliness Clarity and transparency Cultural appropriateness Empathy Variance in experience by service type & nature of complaint Variance in experience by length of time since complaint Variance in experience by client demographics Perceptions of what is working well with awareness materials, legislation, review process, accountability mechanisms, reporting, governance model, resource allocation, change management, etc Clarity of legislative definitions and impact on program Suggested changes in legislation Data Sources / Collection Method Document review Stakeholder interviews Document review Administrative data Stakeholder interviews Client survey Document review Stakeholder interviews Client survey Document review Stakeholder interviews 48

59 Activities (continued) 6. What are the challenges and risks faced in administering and supporting the program and how do these compare to other jurisdictions? 7. What efforts are being made to increase program awareness and how has this impacted the numbers of complaints? 8. To what extent are there role clarity and an effective working relationship between the various program stakeholders? 9. How does the program interact with other related programs? 10. How are program accountability mechanisms functioning? 11. How do the PCQOs report to and communicate with health authority management? Stakeholder perceptions of challenges faced Potential risks identified Strategies used to address risks and mitigate challenges Challenges faced in other jurisdictions Activities completed to raise awareness Adherence to communication directive Clarity of communication materials Trends in the rate of complaints over time and by HA General population program awareness Approaches to raising awareness in other jurisdictions Perceptions of role clarity Perceptions of working relationships and communication between MOH, Secretariat, PCQRBs, PCQOs, Working Group Strengths and limitations of governance model Governance models used in other jurisdictions Description of interaction with other complaints programs such as licensing, Assisted Living Registrar, etc. Perceived degree of overlap vs. complementary relationship Level of clarity of role of each program/process How interactions impact PCQ program operations Opportunities for improvement Perceptions of how well accountability mechanisms are working Degree of rigor in program reporting Suggested success indicators for ongoing program monitoring after the evaluation Description of reporting and communication structures between PCQO and health authority management Document review Stakeholder interviews Client survey Literature review Document review Administrative data Stakeholder interviews Client survey Literature review Future data sources such as PES survey (general population awareness) Plain language review (website) Document review Stakeholder interviews Literature review Document review Administrative data Stakeholder interviews Document review Administrative data Stakeholder interviews Literature review Document review Administrative data Stakeholder interviews 49

60 Activities (continued) 12. How can the program be improved? 13. What other factors or events affected the implementation and administration of the program? Client and stakeholder suggestions for improvements in legislation, awareness materials, review process, accountability mechanisms, reporting, governance model, resource allocation, change management, etc Duplication or overlap that can be eliminated Potential improvements identified in the literature Aspects of program with lowest satisfaction levels Description of other factors or events affecting program implementation and administration Document review Stakeholder interviews Client survey Literature review Document review Stakeholder interviews 14. What are the characteristics of the clients of the program? Demographic characteristics of clients Whether client demographics differ from that of overall population of BC Document review Administrative data Client survey Target Population 15. What is the nature of the complaints received? Service sectors for which complaints are made Nature of complaints made Proportion of complaints received that are out of scope Trends in complaints over time (e.g., complexity, risk level, etc.) Document review Administrative data Stakeholder interviews Outputs 16. What are the key outputs produced by the program? Outputs produced as listed for each program component in the logic model: Policy, oversight and monitoring Awareness Complaint response Document review Administrative data Stakeholder interviews 50

61 Short-term Outcomes 17. What are the outcomes of policy, oversight and monitoring activities? 18. What are the outcomes of awareness activities? 19. What are the outcomes of supporting PCQ Review Boards and PCQOs? 20. What are the outcomes of complaint response? 21. What elements of the program or other factors are most critical in achieving these outcomes? 22. What are the unintended outcomes of the program (either positive or negative)? 23. What other factors or events occur that impact program outcomes? Learnings, advice and best practices are shared province wide Information sharing leads to legislative amendments, policy enhancements and/or system-wide quality improvement, as appropriate Improvements in review process are identified and implemented Increasing public and HA staff/management awareness of health care complaints management process Increasing HA management/staff awareness of roles and responsibilities of the PCQO and PCQRBs Review Boards have the necessary information and supports to adequately review and respond to complaints PCQRBs maintain independence from HAs Improved consistency across Review Boards HA PCQOs, PCQRBs, & HAs respond to complaints & recommendations within mandated time frames MOH responds to recommendations Complainants perceive complaints: process to be clear, consistent, timely, & transparent to be responded to in thorough, respectful & patient-centred way to be effectively resolved HAs identify improvements to quality of care, where relevant PCQRB recommendations lead to improved quality of care Improvements made to reduce future complaints about that issue, where relevant Perceptions of critical elements/factors Critical program elements/best practices identified in the literature Description of unintended outcomes Perceptions of reasons for unintended outcomes Description of other factors or events that impacted program outcomes Stakeholder interviews Document review Future data sources such as PES and staff surveys Administrative data Stakeholder interviews Document review Administrative data Stakeholder interviews Client survey Stakeholder interviews Client survey Stakeholder interviews Client survey Document review Stakeholder interviews 51

62 Appendix C. Data Collection Instruments The following data collection instruments are included on the pages that follow: Stakeholder interview guide PCQ client survey 52

63 Patient Care Quality (PCQ) Program Evaluation Stakeholder Interview Guide Note: This is a general guide only. Each interview will be tailored to the specifics of each stakeholder s involvement with the program. Introduction Hello. My name is from James Murtagh and Associates. We have been hired by the Ministry of Health to conduct an evaluation of the Patient Care Quality Program. As part of this evaluation we are asking for your input during today s interview. The information you provide will be summarized as part of the evaluation report, which will be made available to each health authority. Any information you provide is confidential [to the extent possible in this group setting]. Interviewee names and titles will be listed in an Appendix to the evaluation report. However, specific comments made will not be attributed to you in the evaluation report. Anonymous quotes may be included in the report. Do you have any questions about the evaluation before we begin? Background What is your role with the program? How long have you been in the role? Process How has the program changed since it began/since you have been involved? What was the reason for those changes? Please describe how the program currently operates in your organization/board (PCQOs and PCQRBs: beginning with the first contact with the complainant). How does your organization/board raise awareness of the program? How well do these approaches appear to be working? How does your organization/board interact with and work with other parts of the program? How well is that relationship working? How does the program interact with other complaints programs such as licensing, Assisted Living Registrar, etc.? Is there any overlap or duplication? What type of information do you report or communicate about your program activities? How useful is this information? What do you see as the main strengths of the program as it is currently delivered? 53

64 Prompts, if needed: Awareness materials, legislation, review process, accountability mechanisms, reporting, governance model, resource allocation, change management What challenges or risks does the program face? How are these challenges and risks being addressed? Prompts, if needed: Awareness materials, legislation, review process, accountability mechanisms, reporting, governance model, resource allocation, change management What other factors or events affected how the program functions? Outcomes What learnings, advice and best practices are shared province-wide? How does this information sharing lead to legislative amendments, policy enhancements, system-wide quality improvements, and greater consistency across Review Boards and PCQOs? PCQOs and PCQRBs only: Do you have the necessary information and supports to adequately review and respond to complaints? Do you feel the PCQRBs maintain independence from HAs? Why or why not? How does your complaint review process/pcqrb recommendations lead to improved quality of care? Reduce future complaints about that issue, where relevant? What elements of the program or other factors are most critical in achieving these outcomes? Are there unintended outcomes of the program (either positive or negative)? Other Comments Do you have any other comments on the Patient Care Quality program? Wrap-Up Thank you for your input during this interview. 54

65 Patient Care Quality Program - Client Survey PCQO Version The Patient Care Quality Program deals with complaints about the quality of care you or a family member have received or not received from the BC health system. As someone who made a complaint through this Program, you have been invited to comment on the Program. Many people who made complaints through this Program were also asked to complete this survey. Your feedback will help us improve the Patient Care Quality Program. It will tell us how well the Program is working and help us make the Program better. The survey will take about 5 to 10 minutes to complete. You may ask a family member or friend to help complete the survey. At all times your personal information is protected and will only be used as allowed under the BC Freedom of Information and Protection of Privacy Act. If you have any questions about this survey, please contact the Ministry of Health at. 1. How did you first hear about the Patient Care Quality program? Media (such as newspaper or TV) Pamphlet or poster Health professional (for example: doctor, nurse, pharmacist, or other) Family member or friend Website Other, please tell us Don t remember 2. When did you make your complaint? Month Year Don t remember 55

66 3. What service was your complaint about? Emergency room (for example: emergency visit) Hospital inpatient (for example: hospital stays over one day, surgery) Same day hospital service (for example: blood work, x-rays, cast change or removal, day surgery, minor procedures, IV therapy) Residential care (for example: nursing homes, assisted living homes) Mental health or addictions Home care (for example: nursing care and/or support services received in your home) Ambulance Other, please tell us 4. What was the nature of your complaint? Check all that apply. Medical care (for example: medication, tests, diagnosis) Attitude or behavior of health professionals or other staff Getting services when needed (for example: wait times for surgery, cancellations, getting into a residential care facility) How clear explanations or instructions were (for example: home care instructions, explanations about medical conditions or procedures) Condition of facilities (for example: cleanliness, food services, parking) Leaving the hospital (for example: being discharged too soon or not soon enough) Other, please tell us 5. Why did you make the complaint? Check all that apply. To have my complaint heard To get more information about what happened To have my questions answered To get the service that was refused To receive an apology for what happened To prevent it from happening to other people To have actions taken against the staff or organization responsible Other, please tell us: 56

67 We want to know about the way in which the complaint was dealt with and the result of making the complaint. 6. Thinking first about the way your complaint was dealt with; please tell us how satisfied you were with the following. Check one answer per line. Not at all satisfied Somewhat satisfied Satisfied Very satisfied Don t know Information about how to make a complaint How easy it was to make a complaint Information about what happens after a complaint is made How quickly your complaint was dealt with How thoroughly your complaint was dealt with How fairly your complaint was dealt with How respectfully you were was treated during the complaint process Overall, the way in which my complaint was dealt with If you were not satisfied with any of the above, please tell us why: The following questions are about the result of your complaint. 7.a. Were you made aware of what was done about your complaint? Yes No Not sure b. Are you aware of any improvements that were made as a result of your complaint? Yes No Not sure 57

68 c. How satisfied are you with what was done about your complaint? Not at all satisfied Somewhat satisfied Satisfied Very satisfied Don t know If you were not satisfied, please tell us why you were not satisfied with what was done about your complaint: 8. Did you know that you can contact an independent Review Board if you are unhappy with how your complaint was dealt with? Yes No Not sure 9. Please tell us three things you found helpful about our complaint process: a. b. c. 10. Please tell us three things we could do to improve our complaint process: a. b. c. 11. Do you have any other comments about our Patient Care Quality complaint process? 58

69 We would like to ask a few questions to help us understand the results of the survey. 12. Are you the person who made the complaint? Yes or No 13. Was the complaint about? Your care Someone else s care 13.a. Did you get input from the person who made the complaint while doing this survey? Yes No 14. What is the highest level of education you have completed? Less than high school High school diploma Community college or trades certificate or diploma University degree Other, please tell us 15. What is the language you most often speak at home? English Other language 16. What is your age? Under the age of 18 Age 18 to 64 Age 65 or older b. Was the person who made the complaint the person who received care? Yes No 14. What is the highest level of education completed by the person who made the complaint? Less than high school High school diploma Community college or trades certificate or diploma University degree Other, please tell us 15. What is the language most often spoken at home by the person who made the complaint? English Other language 16. What is the age of the person who made the complaint? Under the age of 18 Age 18 to 64 Age 65 or older Don t know Thank you for completing our survey. Please mail it to us in the stamped, addressed envelope that came with your survey. 59

70 Appendix D. List of Interviewees Ministry of Health Heather Davidson, Assistant Deputy Minister, Health Authorities Division (outgoing, ~April 10) Barbara Korabek, Assistant Deputy Minister, Health Authorities Division (incoming, ~April 10) Teri Collins, Executive Director, Health Authority Relations and Corporate Services Val Stevens, Director, Health Authority Relations and Patient Care Quality Claire Nishimura, Manager, Patient Care Quality Stephanie Constatine, Policy Analyst, Patient Care Quality Doug McDannold, Policy Analyst, Patient Care Quality Leigh Ann Seller, Executive Director, Home, Community and Integrated Care Sue Bedford, Director of Licensing/Community Care Facilities, Home, Community & Integrated Care Robin McMillan, Director, Assisted Living Services Katie Hill, Director, Service Redesign & HCC Christine Massey, Executive Director & Project Lead, Senior s Action Plan Team Silas Brownsey, Executive Director, Senior s Action Plan Team Danielle Prpich, Director, Patient and Client Relations Fraser Health Authority Marlene Grinnell, Chair of the Board Quality Performance Committee Dr. Andrew Webb, VP Medicine Savik Sidhu, Managing Consultant, Client Relations, Quality Improvement & Patient Safety Vancouver Coastal Health Authority Dr. Patrick O Connor, Vice President, Medicine, Quality & Safety Darren Kopetsky, Regional Director, Client Relations and Risk Management Bob Chapman, Director, Client Relations & Risk Management (Vancouver Community) 60

71 Vancouver Island Health Authority Don Hubbard, Board Chair Dr. Lynn Stevenson, Executive VP, People, Organizational Development, Practice and Chief Nurse Laura Nielson, Director, Quality and Patient Safety Provincial Health Services Authority Wynne Powell, Board Chair Georgene Miller, Chief Administrative Officer, Quality, Safety & Outcome Improvement Mary MacKillop, Director, Patient Care Quality Office Interior Health Authority Norman Embree, Board Chair Dr. Jeremy Etherington, Vice President of Medicine and Quality Patty Glaim, Director, Risk Management James Neve, Team Leader, PCQO Northern Health Authority Dale Bumstead, Board Member, Northern Health Authority Michael Leisinger, Regional Director, Board and Administration Services/Chief Privacy Officer Yvette Rutherford, Regional Manager, Patient Care Quality Office Patient Care Quality Review Board Thomas Guerrero, Director, Patient Care Quality Review Board Secretariat Mike Orrey, Manager, Patient Care Quality Review Board Secretariat Dr. John (Jack) Chritchley, Chair, Fraser, Vancouver Coastal, and Provincial Review Boards (Lower Mainland Board) Robert Holmes, Member, Fraser, Vancouver Coastal, and Provincial Review Boards (Lower Mainland Board) Roger Sharman, Chair, Interior Review Board Dr Randall Fairey, Member, Interior Review Board William Norton, Chair, Northern Review Board 61

72 Mr. Richard Swift, QC, Chair, Vancouver Island Review Board Other: Dr. Doug Cochrane, Chair, BC Patient Safety & Quality Council 62

73 Appendix E. Literature Review Report 63

74 Appendix E: Literature Review Report Patient Complaint Systems: Literature Review and Canadian Jurisdictional Scan May 2012 Literature Review - 1

75 Appendix E: Literature Review Report Executive Summary Background: The literature review of national and international information has been included in the evaluation of the Patient Care Quality Program to provide background and context. It also contributes information on evaluation approaches and indicators, strategic directions, and issues encountered. Research questions How have jurisdictions beyond BC (nationally and internationally) structured their health care complaints systems? How have these systems been evaluated or reviewed, what indicators have been employed and what have been the outcomes of evaluations or reviews? Methods: A number of publicly accessible bibliographic databases and internet resources were searched, limited to English language publications from 2006 to February Of the 945 references identified as potentially relevant, 51 were included in the review. Useful documents were also received from the BC Ministry of Health. For the Canadian jurisdictional review, knowledgeable people at the 12 Ministries of Health or Health Authorities beyond BC were contacted by phone in March and April Results International practices: Published information for seven countries was located (Australia, Finland, Ireland, the Netherlands, New Zealand [NZ], the United Kingdom [UK] and the United States [USA]). Common themes and ideas emerged from the literature review of systems in other countries: Patients should understand how to provide complaints and compliments about their care, to both ensure optimal care for themselves and their families, and to help the system to improve. Access to complaints processes should be very clear and user-friendly. This may involve the assistance of a patient advocate / ombudsman to help a patient / family navigate the system. Early on, attention should be paid to acknowledging and clearly understanding the details and nature of a patient s complaint and motivation for complaining; this can lead to managing patients expectations of the system. Some complaints may not be appropriate and processes should be developed to deal with these at an early stage to prevent them from bogging the system down. To as great an extent as possible, complaints should be dealt with at a local level. If an informal local process does not satisfy the complaint, or is not feasible, there should be a clearly identified resource to approach. If this second stage process does not satisfy the complaint, or is not feasible, an independent process should be available to carry complaints to some type of resolution. It was acknowledged that some complaints may never be resolved, despite adequate processes. Comprehensive data should be collected for regular reporting purposes and also for trending; complaint systems should be linked to quality improvement (QI) activities. Standards set by accreditation bodies such as Accreditation Canada and the USA Joint Commission are driving use of patient complaints data for QI. Literature Review - 2

76 Appendix E: Literature Review Report Canadian jurisdictions: In Canada s other 12 provinces and territories (P/Ts), there is variation based on size of the jurisdiction (population and geography), extent of regionalization, maturity of the complaints system, mandate of the provincial ombudsman, etc. All systems encourage local resolution of concerns and have subsequent processes, although these vary. Also, in all jurisdictions, complaints concerning competence of independent providers are managed by regulatory colleges. Several complaint system models appear across the country: Unique solutions: In the Northwest Territories and Nunavut, based in part on small populations Confined to a hospital: In Ontario (no regionalization) and the Yukon (only one hospital) HA-based: In Manitoba, Nova Scotia, Newfoundland & Labrador, and New Brunswick Single health corporation / HA makes it look like a provincial system: In PEI and Alberta Provincial system: In Quebec where the Ombudsman is the final arbiter and Saskatchewan with Provincial Quality of Care Coordinators at Saskatchewan Health who handle complaints not settled at the HA or Ombudsman level Interviews also explored topics like capture of complaints information in databases, use of a severity scoring system, reporting of results, and links between patient complaints and QI activities. The less centralized the system, the less likely is there to be a common data capture /analysis system or linkages to QI activities. With respect to evaluations, it was not clear that any Canadian jurisdictions have conducted formal evaluations of their patient complaints systems. Conclusions: Complaint management systems have long existed at the point of service in health care organizations. Escalation processes have also existed. More formal, statute-based systems / processes are a more recent phenomenon existing in some jurisdictions since the 1990s but still absent in others, including some Canadian jurisdictions. Systems for managing complaints vary in form but all appear to be multi-stage in nature, with initial emphasis on local, relatively informal, resolution. There is some evidence to suggest these local options have been effective in most cases. Local point-of-service options are typically supplemented by one or two more formal options with the final option being an independent or quasi-independent review of the complaint, although the latter may be very limited with respect to types of cases examined. The published literature regarding complaints management tends to be implicitly or explicitly positive but this review found little in the way of empirical research regarding best practice or the overall impact of formal statute-based initiatives in the health sector. To the extent that evaluative literature exists, it appears to be predominantly from Australia, NZ and the UK. There is evidence to suggest complaint systems capture a minority of complaints and that the most formal structures serve an even smaller subset of the population. Awareness and socio-demographic factors are identified barriers to the use of complaint management systems. Challenges linking complaints to QI processes are also evident, as is fragmentation in the complaint process, i.e., the need for patients to deal with more than one process. These challenges persist to varying degrees, even in fourth generation complaint management systems. Literature Review - 3

77 Appendix E: Literature Review Report Table of Contents Executive Summary... 2 International practices... 2 Canadian jurisdictions Literature Search Methods Research Questions for Literature Selection Literature Search Results Literature Search Findings AUSTRALIA FINLAND IRELAND NETHERLANDS NEW ZEALAND UK USA INTERNATIONAL COMPARISONS Canada: Literature & Jurisdictional Review LITERATURE JURISDICTIONAL REVIEW Compliance with Hospital Accreditation ACCREDITATION CANADA USA JOINT COMMISSION Discussion LITERATURE FROM OTHER COUNTRIES FINDINGS FROM WITHIN CANADA Conclusions References Appendix A: Complaint Severity Rating Scale Described in Australia Appendix B: Complaint Severity Rating Scale Described in the UK Appendix C: Balanced Scorecard Approach at the UK PHSO Appendix D: Complaint Severity Ranking System in New Brunswick Literature Review - 4

78 Appendix E: Literature Review Report Literature Review of Patient Complaint Systems in Other Jurisdictions The literature review provides background and context to the evaluation. It also contributes information on evaluation approaches and indicators, strategic directions, and issues encountered. 1.0 Literature Search Methods We searched the following bibliographic databases: PubMed (MEDLINE and non-medline references); The Cochrane Library; the Centre for Reviews and Dissemination (DARE, NHS HEED and HTA); EMBASE; and Web of Science. Searches were run in February 2012 and were limited to English language publications from 2006 to date. Search terms used controlled vocabulary such as the Medical Subject Headings (MeSH) patient advocacy, patient rights, patient satisfaction, quality assurance, health care, and program evaluation and additional keywords such as ombudsman and complaint. Key papers were searched to find related PubMed articles and the reference lists of relevant papers were scanned to identify additional references. Searches for grey literature included NHS Evidence, NLM Gateway, and the New York Academy of Medicine Grey Literature Collection and the websites of several government health ombudsmen or complaints agencies. In addition, useful documents were received from the BC Ministry of Health. 2.0 Research Questions for Literature Selection To ensure that a literature review addresses the topics of most interest to the client, it is ideal to develop specific research questions. These questions guide the selection of literature most likely to be of interest. In collaboration with our MOH clients, the following questions were developed: 1. How have jurisdictions beyond BC (nationally and internationally) structured their health care complaints programs or systems? (Excludes the complaints systems of regulatory colleges and individual hospitals or departments.) 2. How have these complaints programs / systems been formally or informally evaluated or reviewed, what indicators have been employed and what have been the outcomes of evaluations or reviews? 3.0 Literature Search Results The search of bibliographic databases yielded 945 references and searches of the grey literature identified over 600 references (few were directly relevant). The first cut identified 140 potentially relevant references but application of the research questions to article citations/abstracts narrowed the total to 56. These were obtained in full text and reviewed for relevance. Ultimately there were 51 relevant references covering the following jurisdictions (Figure 1): Australia Finland Ireland Literature Review - 5

79 Appendix E: Literature Review Report Netherlands New Zealand (NZ) UK (including Northern Ireland) United States (USA) Reports containing international comparisons: o Australia, Canada, Denmark, Germany, Netherlands, NZ, Northern Ireland, Scotland and Wales o Australia, Taiwan and United Kingdom (UK) Several publications on patient complaint systems in Canada were located (Alberta, Ontario and Quebec). This material will be included in the Canadian jurisdictional review. Figure 1: Modified PRISMA Diagram of Literature Selection Citations identified from the published literature, n=945 Citations identified from a grey literature search, n=600 Total citations / abstracts identified & reviewed, n=1545 Potentially relevant references identified after first cut, n=140 Excluded references, n=89 Potentially references after application of research questions & full text review n=51 Literature Review - 6

80 Appendix E: Literature Review Report 4.0 Literature Search Findings Results are organized alphabetically by country according to a consistent set of headings: history, description, evidence of system evaluation and other. 4.1 AUSTRALIA A number of documents were available for the Australian system, with a key resource being the Australian Commission on Safety and Quality in Health Care (ACSQHC) which provides national guidance. Below the level of guidance provided by the ACSQHC are multiple systems reflecting the way health care is delivered in Australia, i.e., a mix of federal and state responsibilities and public and private systems. History: National policy and practice in complaints is advised by ACSQHC which was established in 2006 (taking over from the Australian Council for Safety and Quality in Health Care [ the Council ]). Before its dissolution in 2005, the Council reviewed research on good complaints management including surveys of 53 health care organisations; consultations; and examination of existing policies, standards and laws. From this grew a report titled Better Practice Guidelines on Complaints Management for Health Care Services. The Guidelines provide practical assistance to organizations wanting to develop or improve their complaints management systems and promote positive attitudes to complaints as part of a wider consumer feedback strategy (ACSQHC 2004 / 2012). A companion document is a complaints management handbook to provide practical information for anyone seeking to use the Guidelines (Australian Council, 2005). These resources appear to still be in use as they were carried forward unaltered by the ACSQHC. Description: A number of organizations have parallel responsibilities for complaints and quality of care at the federal and state levels 1 (Lister et al., 2008). 1. Federal: o o For certain Medicare-supported health services and care for elderly people, relevant resources are the National Aged Care Commissioner and a National Aged Care Investigation Scheme. As stated above, national policy and practice in patient complaints is advised by the ACSQHC. 2. State: o o The Australian Health Care Agreement (AHCA) requires states to maintain complaints bodies independent of public hospitals with powers to investigate and resolve complaints and recommend system improvements; roles and functions of these complaints bodies are set out in state legislation. AHCA also requires a Public Hospitals Complaints Charter setting out how complaints can be lodged. Each state has a Health (or Health & Community Services) Complaints Commissioner. The Commissioner develops its own specific objectives, e.g., the Health Quality and Complaints 1 Complaints directed at providers and organizations appear to be managed by the same process, unlike the dichotomy of processes in Canada where regulatory colleges receive and investigate complaints about providers, e.g., physicians, nurses, technologists. Literature Review - 7

81 Appendix E: Literature Review Report o Commission of Queensland (HQCC) lists its values as respect, integrity, independence, learning and responsiveness, and is also developing a charter of patient rights. Processes vary state-to-state but are generally described as occurring in two steps: (1) Attempted resolution within the practitioner or provider organization, formal or informal. (2) Referral to the Complaints Commissioner for assessment and then investigation and resolution with the provider, or further review if required. Additional action can include referral to another agency such as registration board or prosecution service. System-wide issues may be reported to the State Minister of Health. o Timescales examples are provided by Lister et al. (2008): HQCC Queensland - Informal local resolution = unspecified - Initial assessment = 10 days maximum - Investigation and resolution with provider = 30 days maximum - Further assessment and investigation leading to action = 90 days maximum New South Wales - Overall timetable = 60 days to assess, investigate and decide - Response to all parties = within 14 days of decision - Investigation and resolution = 90 days o Complaints agencies report to State Parliaments on their performance. 3. Private: A high proportion of health care in Australia is insured and provided by the private sector. For private health insurance subsidized by federal rebates (covers 44% of Australians) there is a Private Health Insurance Ombudsman. The Ombudsman s roles and functions are set out in Legislation (the Private Health Insurance Act 2007). 4. Several additional avenues are available for complaints about mental health care (McSherry & Klauber, 2011). Evidence of system evaluation: Bismark et al. (2011) investigated patient satisfaction / dissatisfaction related to complaints about poor quality / quantity of information for informed consent in the state of Victoria (population 5.2 million). They hypothesized that an important explanation for patient dissatisfaction is an expectations gap, i.e., discordance between what complainants wanted and what they eventually got out of the process. Through chart review of 218 suitable cases over 7 years of care ( ), remedies sought and obtained were classified into four categories: communication, restoration, correction and sanction. Results showed a gap between what consumers expected and their levels of satisfaction, i.e., only 34% of complainants who sought restoration received it, 20% of complainants who sought correction received assurances that changes had been or would be made, and only 9% of complainants who sought sanctions saw steps taken to achieve this. The authors suggested that the gap between expectations and outcomes is due to high expectations on the part of patients and / or failure to deliver adequate remedies to complaints. Literature Review - 8

82 Appendix E: Literature Review Report They presented several ways to try and deal with both causes of the gaps: Other: Re: high / unrealistic expectations of patients / families: Make sure front-line staff understand what patients want (try to be attuned their motives for complaining). Wherever possible, the remedies a complainant prizes should be articulated and carefully considered. Re: delivering adequate remedies: Complaints agencies must recognize that, for many complainants, the process will not and sometimes cannot deliver the remedies the patients / families seek, e.g., disciplining a physician. A dose of reality should be administered early in the process to help manage expectations and avoid unnecessary distress and disappointment. Using complaints to improve care: According to the ACSQHC, Increasing implementation of the Australian Charter of Healthcare Rights provides a mechanism to align the complaints received with the rights recognized by the health care system. This alignment offers a good opportunity to use the experience of patients when designing changes to the health care system and its processes. Patient complaints were mapped to the Australian Charter of Healthcare Rights and demonstrated that the largest number of the complaints received related to the rights of safety and communication. An Open Disclosure approach has been adopted to encourage effective and responsive clinician / patient communication, focused on preventing similar incidents (ACSQHC, 2009; ACSQHC, 2010). Assessing complaints according to a severity rating scale: A scale is described that employs a matrix plotting severity of consequences and likelihood of recurrence. Details are contained in Appendix A. 4.2 FINLAND One article was available, although the topic was limited to patient complaints about psychiatric care (Valimaki et al., 2009). History: The right to complain about care and the right to receive feedback (and financial compensation, when appropriate) has been enshrined in European countries since the mid-1990s. In Finland, 20% of complaints involve psychiatric care. Description: Required by law since 1992, all public and private health care institutions must have a patient ombudsman to assist with patient complaints. The ombudsman informs patients of their rights and assists them as necessary in submitting a complaint concerning treatment or a claim for indemnity for professional negligence. This system was envisioned as an easy and flexible system to facilitate complaints and their resolution at the closest possible point of care. The first step is for a patient, with the assistance of the patient ombudsman if necessary, to lodge a complaint with the chief medical officer (the legal authority supervising health services at an institution); a response must be generated within a few weeks. An alternative is to complain to Literature Review - 9

83 Appendix E: Literature Review Report public authorities who have a right or responsibility to monitor the topic that the complaint concerns, e.g., a political health committee or municipal health board. Adequacy of response is monitored by State Provincial Offices overseeing the activities of public and private health care organizations. Patients may complain directly at this level if they are not satisfied with responses at lower levels. State and national bodies oversee quality of care by providers and these bodies have the power to limit a provider s practice (presumably similar to Canada s professional regulatory colleges). A number of other complaint routes are described, depending on the situation: o Ministry of Social Welfare and Health o General representatives of law and order including the Chancellor of Law and Justice or the Parliamentary Ombudsman o Administrative District Court (specifically for involuntary treatment) unsatisfactory outcomes can be appealed to the Supreme Administrative Court o Finnish Patient Insurance Center (can also provide compensation) the patient ombudsman may assist the patient with this process o Committee of Consumer Complaints (for private health care) The ultimate route is a civil suit addressed to the European Court on Human Rights if a complainant is not pleased with the decisions reached by the legal authorities in Finland. Evidence of system evaluation: Not described in the single available article. Other: The multiple avenues for complaints make the task complex for patients, despite the support of a patient advocate. Also, patients are pessimistic about the outcome due to unbalanced patient /provider power structures, they may fear punishment including lower quality of care, they may have trouble writing official complaint letters (despite a patient advocate s assistance at no cost) and it is known that complaints and appeals processes take a long time in Finland. 4.3 IRELAND Five resources were available (Managing patient complaints in Ireland, 2004; Condon 2005; McAuliffe, 2005; Irish Statutes Book, 2006; DHC Ireland, 2008). History: Ireland is able to have a single patient complaints system due to the existence of Health Services Executive (HSE), the country s sole public health care provider. The statutory framework for handling complaints is contained in the Health Act 2004 and its 2006 Regulations which came into force in 2007 (Condon, 2005; Irish Statute Book, 2006). The statutory framework applies to all service providers working on behalf of the HSE, i.e., hospitals and other agencies. Each provider organization can use the HSE system or establish its own if the HSE is satisfied that this is of an appropriate standard. This action grew out of a 2002 study where it was revealed that, in the ad hoc voluntary complaints system in place at the time, few patients complained (7%) or were aware of how to do so (5%), 58% were dissatisfied with the outcome of their complaint, and 68% were dissatisfied with the handling of their complaint (Managing patient complaints in Ireland, 2004; Condon 2005). Literature Review - 10

84 Appendix E: Literature Review Report Description: The 2006 Regulations framework initially seeks to resolve a complaint at a local level by encouraging the complainant to approach the provider or a complaints officer. The complaints officer must work to arrange resolution of the issue and prepare a report for the complainant, provider and HSE within 30 days. HSE or the local chief executive has 30 days to respond to the report s recommendations. If the complainant is dissatisfied with this process, there is access to independent review via a review officer appointed by HSE. The review must take place within 20 working days with a report generated for all parties, either supporting the original recommendations or developing new ones. Following the outcome of this review, the complainant may access the Office of the Ombudsman or the Office of the Ombudsman for Children, regardless of the results of the review. Complaints may be made by persons representing the patient but must be made within 12 months of the actions which give rise to the complaint. Evidence of system evaluation: A formal evaluation of the Irish system was not identified. However, in a comprehensive report on patient safety in Ireland (DHC, 2008), issues related to patient complaints are often mentioned including reference to the fact that the Board of Management of a facility must review, on a regular basis, the systems of governance, including risk management and audit, relating to healthcare safety, quality and performance, including complaints. It further states that groups should be established through which the professional regulatory bodies collaborate on areas of common interest, including complaints management and regular audit of this information. 4.4 NETHERLANDS Three recent articles described the patient complaints system in the Netherlands (Alhafaji et al., 2009; Christiaans-Dingelhoff et al., 2011; Wendel et al., 2011). History: The system focuses mainly on the protection of the rights of the individual complainant, unlike some countries where the complaints system is seen more as a vehicle for improving quality of care, e.g., NZ and the Nordic countries (Alhafaji et al., 2009). Health care reform in 2006 moved to a system of managed competition where insurers negotiate with suppliers about the quality and price of care. Service quality and consumer satisfaction have therefore increased in profile, including management of complaints (Wendell et al., 2011). Description: Informal and formal complaints systems exist; these may be employed separately or several may be pursued at the same time (Alhafagi et al., 2009; Christiaans-Dingelhoff et al., 2011): Informal option: Complaints officials at hospitals (also known as patient advisors, ombudsmen, or advocates) receive complaints, provide information about procedures, advise individual complainants and mediate complaint procedures. Formal options: 1. Complaints committees: The 1995 Clients Right of Complaint Act requires a health care institution to establish a complaints committee composed of at least three people, e.g., an independent chairperson, a health care practitioner, and a representative of a national or regional clients organization. Three objectives are: (1) to restore the relationship between a patient and a practitioner, (2) to improve health care quality and (3) to reduce the number of Literature Review - 11

85 Appendix E: Literature Review Report complaints being lodged under other more time-consuming and costly procedures (as listed below). Time lines are short at 1 to 2 months but decisions are not binding and therefore cannot be appealed. 2. Medical disciplinary boards (five regional, one central): Used for cases pertaining to doctors, dentists, pharmacists, psychologists, physiotherapists, midwives, and nurses, these are composed of lawyers and providers from the profession under investigation. Goals are to promote quality of professional practice and to protect patients. Boards have the power to impose sanctions but time lines are long at 18 months or more. Possible actions are a warning, a reprimand, a fine (maximum 45,000, payable to the State), temporary suspension and permanent suspension. Parties may appeal the decision of a regional board to the central disciplinary board. 3. Civil law procedure: This three-phase process involving the insurer, the conciliation board and the civil court is open to appeal to a higher court and may be lengthy. 4. Criminal court: The goal is protection of the public interest and this action cannot be initiated by the patient. Hospitals maintain databases that capture information on these complaints processes, i.e., the reporting system of the complaint officer for informal complaints, the reporting system of the complaint committee for formal complaints, and the reporting system of the liability officer for medico-legal claims. Evidence of system evaluation: Alhafaji et al. (2009) refer to a 2002 official evaluation based on the opinion of lawyers, judges and panels of patients and health professionals; however, the specific experiences of complainants and practitioners against whom a complaint was made were not examined. Wendel et al. (2011) reported on a postal survey of 186 members of a Dutch insurance panel who lodged a complaint to the insurer within the previous 12 months; 53% related to inadequate compensation. Twenty-one Likert-scale-type survey questions assessed functional quality (how the service is delivered, e.g., friendliness, fairness, responsiveness), technical quality (what is delivered and adequacy of outcome) and satisfaction / trust. Response rate was 82% (150 people). The results showed: o A positive effect of functional and technical quality on complaint satisfaction (p < 0.001) o Significant effects of functional quality on trust in the company (p < 0.001) o Significant effects of technical quality on trust in the company (p = 0.011) o Functional quality significantly affects overall satisfaction (p < 0.001) o Lack of significant effect of technical quality on overall satisfaction Other: The Dutch government is proposing strengthening patients rights through modification of existing legislation or possible enactment of new patients rights legislation in a Client and Quality of Health Care Act. The goal will be to create a more effective and easily accessible complaints procedure with complaints resolved at the earliest possible stage (Alhafaji et al., 2009). Literature Review - 12

86 Appendix E: Literature Review Report 4.5 NEW ZEALAND A number of documents were available to describe the patient complaints system in NZ, i.e., Bismark et al. (2006a), Bismark et al. (2006b), Lister et al. (2009), Temelkovski & Callaghan (2009), Manning (2010), HDC (2011), and HDC (2012). History: The NZ system pre-dates those of many other countries, being based on a compensation system introduced in the 1970s for workers and motor accident victims. It is the envy of world leaders in the field of patient safety and medical error who struggle with the modest goal of introducing reforms to or alongside the tort action to make it perform more adequately as a mechanism for compensating patients, doing justice between the parties, and preventing accidents in the health system (Manning, 2010). The focus is on no fault compensation which bypasses adversarial case-by-case litigation (Temelkovski & Callaghan, 2010). Patient rights were enshrined in 1994 legislation, the Health and Disability Commissioner Act (HDC, 2012). The Code of Health and Disability Services Consumers' Rights and an independent nationwide advocacy service and Health & Disability Commissioner (HDC) complaints system began in 1996; the Code includes the right to complain and have the complaint taken seriously (HDC, 2011). Description: Patients with complaints can take one of two distinct paths depending on their interest in monetary or nonmonetary remedies (Bismark et al., 2006): Monetary compensation claims go to the Accident Compensation Corporation (ACC) which awards compensation on a no fault basis. Anonymized claims data are analyzed and shared with providers to support improvements in patient safety. Non-monetary cases go to the central, independent HDC which resolves complaints by advocacy, investigation or mediation; acts as a gatekeeper to disciplinary proceedings; and disseminates findings so that lessons can be learned. Within the HDC is a national network of independent advocates who report to a Director of Advocacy and an independent prosecutor, the Director of Proceedings. As the agency s mandate includes health care quality, it can make recommendations even when no complaint has been raised. Local handling of disputes will usually start within the provider organization in one of the 21 District Health Boards (DHBs). Customer Relations Officers at a DHB provide the first point of contact for complainants, with an option to contact the HDC (Lister et al., 2009). A variety of dispute resolution routes are available to the HDC (Manning, 2010; HDC, 2012): o No further action (the Commissioner considers the complaint unnecessary or inappropriate or is satisfied that there is already an adequate remedy). o Alternative disputes resolution procedures (patient advocacy and mediation). o Referring the complaint back to the provider to resolve. o Referring to other agencies (e.g., practitioner s registration authority) for a competence or fitness-to-practice review. Literature Review - 13

87 Appendix E: Literature Review Report o Formal investigation by the HDC for potentially major breaches of the Code of Rights such as allegations of sexual impropriety and of lapses in that resulted in death or significant disability. The website of the HDC appears to be very user-friendly with a web link to report complaints displayed prominently at A patient brochure explaining the process is available in nearly 50 languages. Evidence of system evaluation: Lengthy and detailed annual reports for the past 15 years are available on the HDC website and report on progress made towards a climate of complaint resolution and accompanying system improvements. Case studies are provided. Evidently user surveys are done periodically, e.g., the 2011 Annual Report states that survey results from complainants and providers who have participated in our process tell us that 81% agree that their complaints were taken seriously and 73% were satisfied that we had managed the complaint well. In addition, an evaluation of the NZ HDC appears to have been done in 2010 or 2011 and rolled into a Statement of Intent, in both full and easy-to-read versions (HDC, 2011a; HDC, 2012b). The Statement of Intent provides information against which the organization s performance can be assessed, about how the HDC uses resources and exercises its statutory powers, and how it intends to make a difference to the lives of New Zealanders (HDC, 2012a). Other: The relatively long history (15+ years) of the NZ complaints system lends itself to research using the various databases capturing complaints data, e.g.: Bismark et al. (2006a): The authors estimated the proportion and characteristics of patients with adverse events (AEs) in NZ public hospitals in 1998 who complained to the HDC. Data linkage between the HDC Complaints Database and the NZ Quality of Healthcare Study Database allowed for comparisons of characteristics of complainants (n=348) and non-complainants (n=847). Results showed that 0.4% of patients suffering AEs complained, rising to 4% for serious AEs. Odds of complaining were significantly lower for patients who were elderly, of Pacific ethnicity or who lived in the most deprived areas, suggesting troubling disparities in access to and utilization of complaints processes. Bismark et al. (2006b): The authors were interested in the forms of accountability sought by injured patients using HDC complaints data (n=254 letters) but also ACC data (n=582 patients). Sociodemographic and socio-economic characteristics of patients who sought nonmonetary forms of accountability were compared with those of patients who claimed compensation. Results showed that, of 154 injured patients whose complaints were sufficiently detailed to allow coding, 50% sought corrective action to prevent similar harm to future patients (45% system change, 6% review of involved clinician s competence) and 40% wanted more satisfying communication (34% explanation, 10% apology). The odds that patients would seek compensation were significantly increased if they were working age (30 to 64 years) or had suffered a permanent disability. In contrast, when injuries resulted in death, the odds of a compensation claim to the ACC were about one-eighth those of a complaint to the HDC. The authors concluded that compensation is important to some, especially when economic losses are substantial however, others have purely Literature Review - 14

88 Appendix E: Literature Review Report nonmonetary goals and ensuring alternative options for redress would be an efficient and effective response to their needs. Temelkovski & Callaghan (2010): The authors mapped 100 HDC complaint investigations to the six Domains of Competence for physicians as set out by the Medical Council of NZ, i.e., medical expert, communication, collaboration, management, scholarship and professionalism. 2 Results showed that the most common areas of competence identified as problems were medical expert (93% of cases) and communication (48% of cases), with an average of 1.8 domains per investigation. 4.6 UK Many references reported on the patient complaints system in the UK including several specific to Northern Ireland (DHSSPSNI, 2009a; DHSSPSNI, 2009b) and Scotland (Gulland, 2006). However, since the patient complaints system in the UK underwent significant reform in 2009, only resources from 2009 forward are summarized in this section. The key document was the 40+ page document Listening, Responding, Improving issued by Department of Health and described as a best practice guideline (DOH, 2009). History: In 2006, the National Health Service (NHS) committed to reform of the patient complaint system; this became a reality via Regulation on April 1, 2009, after 90 organizations pilotted the system. A single approach was introduced for dealing with complaints in health and social care services, including complaints made against health professionals. The aim is to encourage a culture that proactively seeks and then uses care experiences to improve quality. The new system was developed after extensive background research described a system that was inadequate, inflexible, too complex and slow to respond (DOH, 2009). Description: The NHS Constitution (2010) states that individuals have the right to: Have any complaint they make about NHS services dealt with efficiently and have it properly investigated. Know the outcome of any investigation into their complaint. Take their complaint to the independent Parliamentary Health Service Ombudsman (PHSO) if they are not satisfied with the way the NHS has dealt with their complaint. Make a claim for judicial review if they think they have been directly affected by an unlawful act or decision of an NHS body. Receive compensation where they have been harmed by negligent treatment. Complaints are dealt with via the same system regardless of whether financial compensation or health providers actions are involved. All NHS provider organizations have legal, contractual and professional obligations to provide an accessible, responsive complaints process for service users. The process involves first developing systems to receive complaints and even proactively seeking patient feedback (complaints and compliments) these must be in compliance with the 2009 Regulation (MPS, 2011). 2 The Royal College of Physicians & Surgeons of Canada has employed a similar system (CanMEDs) since 1996 with seven roles that lead to optimal health and health care outcomes: medical expert (central role), communicator, collaborator, manager, health advocate, scholar and professional. Literature Review - 15

89 Appendix E: Literature Review Report In physicians practices this involves appointing (a) a practice partner to act as a responsible person and (b) a complaints manager (these may be the same person). The severity of the complaint (low, medium or high) is assessed using a risk assessment tool. A number of expectations must be met at this point, i.e., acknowledging complaint receipt within 3 days; discussing with the complainant how their complaint will be handled; a written plan for resolution including timeline, progress reports, support and follow-up; etc. (MPS, 2011). In addition, a Patient Advice and Liaison Service (PALS) is available to provide information about the NHS including: details about how to complain, help to resolve problems, referrals to support groups and agencies (DOH, 2009). An alternative to complaining locally is to use a free Independent Complaints Advocacy Service offered by non-profit organizations like the Carers Federation Ltd. This is considered most appropriate if: Concerns were raised with providers but not fully resolved Serious questions about standards of care arose The case involves complex issues that require investigation The issues involved concern more than one organization (Carers Fed Ltd., 2010). Most complaints are resolved locally but complainants must be advised of their right to ask the PHSO for a review if they are unhappy. The PHSO is the second and final point of contact for complaints within the NHS (Wise, 2010). Complainants must provide reasons why they are dissatisfied and consider whether they have suffered hardship or injustice (MPS, 2011). The PHSO will only take on a complaint after resolution has been attempted with the organization involved. (PHSO, 2010; PHSO, 2012). The PHSO process involves receiving the complaint, assessing its appropriateness for PHSO investigation, and determining that local processes have been adequately pursued. If these steps are complete, the PHSO then conducts a more in-depth assessment to ensure the complaint relates to administrative fault or service failure and that injustice or hardship has flowed from it. If the PHSO is unable to resolve the issue, a formal case-by-case investigation is conducted with an aim to complete the process within 12 months (PHSO, 2010; PHSO, 2012). Evidence of system evaluation: Organizations, including provider organizations, are required to produce annual reports documenting number of complaints received, whether complaints have been upheld, number of cases referred to the PHSO and the issues that complaints raised including lessons learned and actions taken (DOH, 2009). The PHSO also produces a detailed annual report including statistics and case studies (PHSO, 2011a; PHSO 2011b). From the amount of literature available about the new patient complaints paradigm in the UK, it appears that a great deal of energy is being invested. Strategic plans aim to continuously improve the system, i.e., a plan developed by the PHSO aims for accessibility, excellent customer service and justifiable decisions and recommendations (PHSO, 2011c; PHSO, 2011d). The PHSO uses a balanced scorecard approach (40 performance indicators of which 10 are considered to be key ) is used by the PHSO to assess its performance (see Appendix B). Literature Review - 16

90 Appendix E: Literature Review Report Other: Northern Ireland (NI): A statutory patient complaints framework was outlined in the 2004 Health Act and detail about a two-step patient complaints management process was established as of April 1, The second step, after local resolution has failed, is referral to the NI Commissioner of Complaints (the Ombudsman ). This single-stream model covers hospital and community services, providers, publically funded staff in private-pay facility beds, the blood transfusion service, the Public Health Agency, etc. (DHSSPSNI, 2009a; DHSSPSNI, 2009b). Assessing complaints according to a severity rating scale: As in the Australian materials, a complaint severity categorization scale is described employing a matrix that plots severity of consequences and likelihood of recurrence. Details are contained in Appendix B. 4.7 USA A number of documents were reviewed to compile brief information on patient complaint systems in the USA 3 (VHA, 2005; Pelling & Spath, 2006; Thacker, 2006; VHA, 2006; Medicare Part D, 2008; Pichert et al., 2008; ECRI, 2010; Birmingham, 2011; Troyer & Sause, 2011). An interesting paradigm in the USA is to distinguish between complaints and grievances where complaints are defined as minor patient issues that can be resolved within 24 hours and involve staff members who are present at the time of the complaint, e.g., housekeeping or meal issues, and grievances are more substantive, being unresolved complaints or more in-depth concerns. Centers for Medicare & Medicaid Services (CMS): CMS requires providers to formally notify beneficiaries (inpatient and outpatient) of their right to communicate concerns about the quality of the care they received to the state Quality Improvement Organization (QIO) in order to raise awareness about the right to complain. As an impacted manager stated, Look for ways to empower patients to provide feedback, good and bad, and make it easier for patients to contact you about quality concerns. Find out what the patients are concerned about and act before they feel the need to take their complaints further. Remind patients at every opportunity that you care about them. (Pelling & Spath, 2006; Birmingham, 2011). An evaluation / audit was available for management of complaints and grievances for Medicare Part D (covers 26 million lives) (Medicare Part D, 2008). o o Complaints: The monthly complaint rate decreased by 74% over the 18-month period of study and the mean time to resolution decreased from 33 to 9 days. However, ongoing implementation issues were noted, mainly related to enrollment information processing. Grievances: Reporting was less complete and findings were limited, i.e., it was not possible to determine whether beneficiaries who filed grievances were at risk of depleting their medications or whether plan sponsors were resolving grievances in a timely manner. Overall, 3 Note that, due to differences in medical care and multiple systems across the country, USA models may not be of great relevance for Canada. Literature Review - 17

91 Appendix E: Literature Review Report data anomalies raised questions about whether plan sponsors were reporting consistently and accurately. o CMS conclusions and actions: The auditors concluded CMS s oversight efforts have focused almost exclusively on resolving complaints with little attention devoted to plan sponsors grievances processes Several plan sponsors indicated that [CMS] guidance was insufficient, increasing the likelihood that plan sponsors report erroneous and inconsistent information to CMS and that they rely on the wrong processes to address beneficiaries concerns. ECRI (2010): Based in Philadelphia, the ECRI Institute is an independent, non-profit organization that researches the best approaches to improving the safety, quality, and cost-effectiveness of patient care. In its Healthcare Risk Control series, ECRI produced a report called Managing patient grievances and complaints. The report states that health care organizations must have well-defined processes for addressing and responding to patient grievances and complaints, as required by the major USA payers, CMS and Veterans Health Administration (VHA). The ECRI document provides a set of recommendations on desirable processes (Table1). TABLE 1: RECOMMENDATIONS FOR A PATIENT GRIEVANCE PROCESS (ADAPTED FROM ECRI [2010]) Develop and implement well-defined policies, procedures, and processes. Be aware of related regulations and standards from CMS, the Joint Commission, and other important entities. Interview patients and family members or other representatives to determine whether they understand the grievance process. Ensure that the facility s governing body approves and oversees the grievance process, unless they assign these duties in writing to a grievance committee. Inform patients of their rights upon admission or as soon as possible after admission and include information regarding their right to file complaints or grievances. Establish appropriate turn-around times for responding to grievances. Ensure that responses are written in clear and easily understandable language and include information about what actions were taken to resolve the issue, etc. Educate all staff members, especially those with direct patient contact, on grievance processes; emphasize that staff should communicate calmly and show empathy for patients concerns. Collect, track, and trend data on grievances and complaints as part of QI. Consider implementing a patient advocate program if one is not already in place. Nursing homes / long-term care: In the USA, two agencies receive and investigate nursing home complaints, the state certification agency (which also does annual inspections) and the state Long- Term Care Ombudsman Program. North Carolina researchers explored nursing home complaints using these sources in their state and reported that the roles and processes of the two agencies differ significantly with little cross-over in complaint receipt (Troyer & Sause, 2011): o The Division of Health Service Regulation (DHSR; the local state certification agency) follows CMS protocols and guidelines with nursing homes inspected annually. Between surveys, Literature Review - 18

92 Appendix E: Literature Review Report complaints made by the resident, family, staff, or public may be investigated. The complaint process is often difficult to navigate, particularly by residents with limited functioning and a lack of resources. The DHSR can access medical records for complaint validation and also access areas of the facility that are generally off-limits to the general public to assist in their complaint investigation. The DHSR may therefore rely on document review for proof of a complaint and only considers complaints that fall within the purview of the federal regulations for nursing homes set forth by CMS. With respect to action, the DHSR may cite a facility with a deficient practice. The facility is then required to develop and implement a plan of correction. o The state Long-Term Care Ombudsman Program was established in 1978 as a result of the Older Americans Act (OAA), primarily to improve nursing home residents chances for fair consideration, due process and choice. State and local area ombudsmen supervise paid program staff as well as volunteer ombudsmen (8700 nationwide) who are trained and certified to deal with residents complaints (Thacker, 2006). A representative visits each nursing home at least quarterly either as a friendly visit, a training visit to educate residents and staff members or a complaint/mediation visit to assist in complaint resolution (Troyer & Sause, 2011). Frequent exposure means residents and staff members are comfortable with the representatives and have greater opportunities to file complaints. Unlike DHSR, the Ombudsman does not have a specific scope in which the complaints must fit. However, the Ombudsman is limited in access to information, i.e., must have a court order or resident permission to access medical records and does not have access to some areas, such as the kitchen. As a result, the Ombudsman may consider interviews with staff and residents more significant than document review. With respect to actions, the Ombudsman has flexibility in addressing complaints, i.e., may mediate between the complainant and the administration or staff or provide staff training and education to address the issue (Troyer & Sause, 2011). Information about residents complaints about 200,000/year has been collected through the National Ombudsman Reporting System (NORS) since Nationally aggregated and statelevel annual complaint data are publicly available and accessible on-line for use by government officials, state and local level ombudsmen, academic researchers, the long-term care industry and the general public to gain a sense of the type of improvements in long-term care that need to be made to improve residents quality of life (Thacker 2006). Veterans Health Administration (VHA): Patient Care Advocacy Programs are required in all VHA facilities to address patient complaints (VHA, 2005). In addition, patient complaint data (classified according to a nation-wide set of complaint-specific codes) must be used in medical center committees to represent patient views and to identify and report trends. The Advocacy Program must be promoted through posters in high-traffic areas, brochures and media, and an Advocate must be available when needed. Response to complaints is required within 7 days. Appeals to decisions made at the facility level move to the VHA Network where an internal review is conducted or an impartial review by a professional board external to the agency may be arranged with a timeline of days (VHA, 2006). An evaluation of the VHA complaints system was not identified. Literature Review - 19

93 Appendix E: Literature Review Report 4.8 INTERNATIONAL COMPARISONS Two documents provided international comparisons of patient complaint systems (Lister et al., 2008; Hsei 2011). Lister et al. (2008): Produced by a Danish consulting company for the National Audit Office in the UK, this report supplied background, lessons for the UK, and best practices to prepare for a single comprehensive complaints handling system in England. Countries chosen for comparison were Northern Ireland, Scotland/ Wales, Australia, Canada, Denmark, Germany, NZ and the Netherlands. These countries were selected as being the most likely to be relevant to the UK. Following review of the nine complaints systems, key themes and lessons were grouped under 14 headings (also see Table 2): Organizational responsibilities stress resolving complaints informally with the care team as much as possible. Ideally, complaint support staff members train front line staff about how to deal with complaints. The complaint appeals organization must be independent with the most important functions being an appeals process and follow-up actions resulting from complaints, particularly where they raised system-wide issues. Legislation and regulation should provide a basis for patient/client rights and staff expectations (standards or guidelines) and outline a clear simple process. Objectives should translate the ideals and values of service providers and complaints agencies into performance measures, encouraging feedback from staff and users to improve services. Definitions of complaints must be broad but measurable (including severity assessment). Handling complaints alongside AEs and improvement suggestions and complaints from staff provides a common basis for learning and improvement. Processes must be simple and avoid hand-offs between different agencies. In most cases, local resolution (informal and formal) supported by independent review processes undertaken by a second agency seemed appropriate. Timescales should be as short as practical. Informal resolution and apology should be immediate where possible. Local formal resolution should be within 4-5 weeks. Timescales beyond this should be short but related to the complexity of the issue. Where policy issues are raised, separate longer term review may be required. Staffing: Most directly involved in local complaints are front line care teams supported by a local complaints team. Both complainants and people who are the subject of complaints should have access to independent support staff or volunteers to help them. Special arrangements are required for vulnerable groups, e.g., children in care and people with mental health problems. Numbers of complaints do not reflect opinions of services. It was notable that where a positive attitude was taken, complaint numbers increased. It was also apparent that some complaints do not reflect a realistic understanding of what patients should reasonably expect and it was seen as important to be able to close such complaints without overloading the system. Literature Review - 20

94 Appendix E: Literature Review Report People often find it difficult to initiate complaints. Obstacles include: perception that the process is long and complex, perception that no action would be taken anyway, and /or fear that there might be repercussions. People most often want an explanation, an apology, and reassurance that the same thing would not happen to others. Patient / client based organizations were seen as helpful. Attitudes of staff (fear of blame) are the most important barriers to positive approaches to complaints. Required are a supportive management culture and greater clarity as to what can reasonably be expected. Apology remains difficult for staff. It requires that apology and resolution of complaints should be separated from acceptance of liability and blame, as far as is possible. Redress: Small gifts may be appropriate to recognize costs and inconvenience, particularly when accompanied by explanation and apology, e.g., one complaints team accompanies a personal apology and explanation with flowers to thank the complainant for helping them improve the system. Monitoring should occur at local and national levels and should involve real examples rather than just statistics, de-emphasize time scales to the exclusion of other outcomes. A national annual report on complaints and QI is helpful. Learning from complaints should occur at local and national levels. Best practice at the local level saw complaints support staff working closely with QI teams. Where national complaint support agencies do not have a broad remit they need strong links to QI and standard-setting agencies. Improvement suggestions were proposed to increase the responsiveness and accessibility of complaints systems and their efficiency and timeliness. Comments included calls for a positive approach to complaints, increased transparency, and keeping complainants informed of progress. It could be helpful if complaints agencies and staff could share their experience through an international conference and on-line community. Hsei et al. (2011): After reviewing the patient complaints systems in the UK and Australia, Hsei contrasted Taiwan s model which was established as an element of hospital accreditation in 2007, i.e., hospitals must establish patient complaint systems and document their responses to complaints. In addition, the Taiwan Department of Health has drafted a Medical Disputes Handling Act demonstrating that the government has acknowledged the issue of patient complaints. Hsei s review showed that Australia and the UK aim to incorporate patient complaints into their national quality systems as a bridge between complaints management and quality management Literature Review - 21

95 Appendix E: Literature Review Report 5.0 Canada: Literature & Jurisdictional Review 5.1 LITERATURE Three articles were located for Canada: Alberta (HCQA, 2007), Ontario (Rogers & Vilhena, 2005), and Quebec (Clement & Gagnon, 2006). Although this material may be superseded by the information obtained in the jurisdictional review below, it is briefly presented here for completeness Alberta: Prior to May 2008, nine regional health authorities (RHAs) and three provincial mental health entities delivered health care; these merged to form Alberta Health Services. In 2007 a provincial patient complaints framework was developed by the Health Quality Council of Alberta (HCQA) to assist RHAs it is a reference piece now. Guiding principles and suggested operational policy statements are provided, as is information on integrating the experience into QI and continued learning and public reporting. The suggested multi-step process to address complaints includes: (a) resolution at the initial point of contact, (b) a formal internal process, (c) an internal Appeals Committee, (d) use of the Alberta Ombudsman as needed, although this would be limited to review of the complaints managing process or the fairness of the decision, i.e., it is not an appeal of the final decision (HQCA, 2007) Ontario: This paper proposes that Ontario hospitals work with the University Health Network (UHN; three large teaching hospitals in Toronto) in a voluntary and collaborative process to capture and compare ( benchmark ) data about patient complaints. Detailed UHN data and information are provided, e.g., processes, workload and timelines. The authors concluded that the challenge then appears to be to get started, and this requires a willingness and formal commitment by health care facilities to willingly and openly participate in the process of benchmarking. There are problems to overcome to establish a scientifically rigorous methodology for comparison. There are also valid organizational concerns and fears to overcome. However, we have an opportunity to voluntarily share data and ultimately support an integrated patient experience (Rogers & Vilhena, 2006) Quebec: Complaint Assistance and Support Centers were established in the early 1990s to respect the rights of users and improve service quality. This Ministry-of-Health-funded study examines the impact of Centers on the intake of complaints. 4 Results showed that the system is accessible and that complaints are processed. However, the system was reported to be hindered by differences and conflicting interpretations among stakeholders as to complaint legitimacy, respect for users rights and the mission of the system (Clement & Gagnon, 2006). 5.2 JURISDICTIONAL REVIEW The experience with Patient Care Quality Systems in Canadian provinces and territories (P/Ts) beyond BC was explored via or phone contact with leaders in patient quality / safety in each jurisdiction. All contacts responded, either with direct information or with advice about others to contact within the P/T (Table 2). Ultimately, knowledgeable people were located within all Ministries / Departments of Health or Health Authorities (HAs). Their contact information is contained in Table 2 for further communication, as required. 4 The study also evaluated whether CAAPs had adequate funding but these data are not reported. Literature Review - 22

96 Appendix E: Literature Review Report TABLE 2: RESULTS OF CANADIAN JURISDICTIONAL REVIEW P/T Name Response Evaluated? AB Pat Brooks, Patient Concerns Officer & ED Patient Relations, Alberta Health Services; bertahealthservic es.ca Eileen Bell, Director of Patient Relations, Calgary Zone of AHS; tahealthservices. ca History: o 2006: A Patient Concerns office was formed in Calgary with 2 staff for the RHA policies and procedures were developed by a working committee with stakeholder consultations Patient Advocates (now called Patient Concerns Consultants) were hired to liaise with patients o 2006: Legislation requires all HAs to develop a Patient Concerns resolution process (Alberta Regulation 124/2006) o 2007: Health Quality Council produced a framework to assist HAs in developing processes; 2008: AHS was formed now finalizing policies for AHS First stage in complaints: Patients are encouraged to address concerns directly with providers (local resolution). Staff members and patients are aware of processes available via posters / brochures on units. Concerns that are not resolved at this level move to the Unit Manager (educated and mentored by the Office of Patient Concerns) and beyond this may move to the Director and VP levels. Central intake at the AHS Office of Patient Concerns is an option if patients choose not to take the local route or if the issue is not resolved at this level. (Also, the AHS website allows for completion of a feedback form for the Office or direct contact with the Office.) The AHS Office of Patient Concerns gets involved via requests from patients or Unit Managers. The Patient Concerns Consultant gathers information from all parties and clarifies what is still outstanding. Lack of resolution at this stage moves the process to the AHS Director of Patient Concerns (Pat Brooks). Lack of resolution may result in referral to the Alberta Ombudsman who started taking on health care concerns in The Ombudsman s Office will only take on cases (must be in writing) after all avenues with AHS have been completed. Focus is on administrative fairness. Issues related to independent physicians providing care in an AHS facility are dealt with by AHS although the College may also be involved. AHS has a disclosure policy requiring physicians to disclose issues/errors with a patient. The matter may undergo a quality review via the Quality Committee. Resulting recommendations may be shared with operational leads in relevant areas, e.g., revisions to the Surgical Checklist. The AHS Quality & Healthcare Improvement Department (Director is Susan Mumme) receives info about complaints currently via informal personal processes but a more formal process is being established via a Safety Learning Reporting System. For complainants interested in legal recourse or interested in media exposure, Patient Relations will stay engaged (there is no Risk Management Department) and often are able to diffuse the issue. This changed in 2006 pre-2006, Patient Relations would bow out of the case. Information for AHS Office concerns is entered into a single database as of 2010 (Datix) report in writing quarterly to the AHS board. Definitions have been provided although a severity scale for complaints is not employed considered it at the outset and may re-examine. No but considering it for 2012 SK Provincial Quality of Care Coordinators: Denise Grad (dgrad@health.g ov.sk.ca) & Susan Burns (sburns@health.g ov.sk.ca) There has been a provincial system for > 15 years: First step is to bring concerns up with the provider or a supervisor information is on RHA websites / pamphlets / follow-up of those expressing concerns on surveys / those referred back if they make contact centrally. If this is insufficient, complaints go to an RHA Client Rep / Quality Care Coordinator learns the story, helps the patient navigate the system, may act as patient advocate, investigates and suggests or enacts solutions. Response to receipt of the complaint is expected in 2 days. If patients are not satisfied with the outcome of complaint investigations, they may appeal within the RHA (every RHA has an appeal process but they vary). Provincial ombudsman can be approached although the scope of cases taken is narrow and a small number moves forward (re: fairness of treatment). There are two Provincial Quality of Care Coordinators at Saskatchewan Health who handle the small number of complaints not settled in the RHAs and not suitable for the Ombudsman (<10/year) re-examine the case and generally are able to solve the problem. The MOH has generated complaints handling guidelines (RHAs develop local policy) and a manual (upgraded in 2010). Also offers an orientation program to teach RHA staff how to deal with concerns, share information with the MOH and interact with an MOH database. Exceptions may go to litigation a few cases are never resolved; complaints specific to a professional go to the regulatory college. RHAs enter all their complaints data into a database (includes types of complaints according to specified categories, program area, outcome/action, response to resolution, time to resolution, etc.). Aggregate information is viewed by the MOH and an Annual Report is generated shared with program areas at MOH to look at types of concerns, trending, specific initiatives (e.g., suicides while in care, falls). RHAs report routinely to their boards MOH facilitates meetings with Patient Care Coordinators twice annually with speakers / networking; also manages a listserv that draws on RHA No Literature Review - 23

97 Appendix E: Literature Review Report Coordinators experience or connect with each other via the MOH Coordinators. MB Name Response Evaluated? Ms. Jean Cox, Acting ADM, Regional Programs and Services Manitoba Health; There is no centralized patient complaint resolution process; however, in June 2011, The Regional Health Authorities Amendment Act (Accountability and Transparency) received Royal Assent. The Act contains a number of amendments that will come into force when proclaimed to enable regulations, guidelines and procedures on a number of topics; one of which is addressing patient concerns. A regulation will include requirements for all RHAs to ensure that a process is in place to resolve patient concerns, that the process is accessible, and that it provides a timely response to concerns. Each RHA currently has a patient concerns process, although some are more robust than others, e.g., the Winnipeg RHA website encourages feedback about care, first at the point of care and then by communicating with an RHA Client Relations Coordinator who is committed to returning contact within 3 days. Information about further steps is not available on the WRHA website. No ON Dr. Ben Chan, CEO Health Quality Ontario; ben.chan@hqont ario.ca Bill 46, the Excellent Health for All Act, 2010 (June 2010) 5 requires each hospital (1) to conduct a patient/client/caregiver (and employee) survey at least annually to assess satisfaction, and (2) to develop a transparent patient relations process Aside from these requirements there doesn t appear to be a province-wide strategy or consistency to the patient complaints process, neither is there an obvious process per Local Health Integration Network (LHIN) (a form of regionalization unique to Ontario). Materials for the University Health Network (UHN; three major teaching hospitals in Toronto) were explored as an example. UHN has a Patient Relations Office / Hospital Ombudsman that handles patient complaints via mediation, conflict resolution and option building ; provides consultation and educational services for staff and physicians; and advocates for system change. The UHN Ombudsman 2009/10 Annual Report states that by policy, all front line staff and physicians are encouraged to try to resolve patient / family concerns as they arise. However, if the patient / family member does not feel that their concerns have been adequately addressed or when there is a recognized need for additional conflict resolution support, accessing the Patient Relations Department is encouraged. The office receives about 1100 complaints annually with mean resolution taking 9 days (62% in 7 days). About 1% of cases were shared with the CEO and 1% proceeded to legal routes. There was no indication of a secondary procedure or appeal route. As detailed by recent media stories, the Ontario ombudsman does not have the authority to examine hospital affairs.? QC Dr. Robert Salois, Health and Welfare Commissioner of Quebec; robert.salois@cs be.gouv.qc.ca One system handles complaints about health care & social services: 6 The first step is via the local Complaints and Service Quality Commissioner of the facility delivering services (has 45 days to respond). Every facility must also have a Users Committee that can act as a resource. A regional Complaints and Service Quality Commissioner deals with issues for community organizations, private nursing homes, residences for the elderly, ambulance services, and health and social services agencies (has 45 days to respond). Complaint Assistance and Support Centres are regional community organizations (free and confidential) mandated by the Minister to assist and support users, as set out in legislation. Second recourse is via the Quebec Ombudsman who accepts cases where the complainant is dissatisfied with the conclusions of the Complaints Commissioner or a response from the Commissioner has not been received within 45 days. Complaints related to physicians, dentists and pharmacists are not eligible these move from the local Commissioner to a medical examiner system that also has a back-up review procedure. Private facilities are not covered by the Ombudsman. There is also a Complaints Assistance Center available by phone province-wide (although web information is only in French). No? 5 Available at: 6 Information from the MOH Quebec Ombudsman s website at: & a web-based document called Health and social services complaints examination procedure available at: Literature Review - 24

98 Appendix E: Literature Review Report NB Name Response Evaluated? HA patient reps: There is no provincial system (although there is interest in developing one); however, processes exist in each of the two HAs in NB. The complaint No Kathleen Snow systems handle acute health care complaints (excluding self-employed providers). Hospital units have brochures and the HA websites include (Horizon); information describing the complaint system, especially the availability of Patient Representatives. The first step is for a patient with a concern to discuss the issue with a provider, especially a nurse manager (most have had education and zonnb.ca mentoring from Patient Reps). Renee-Claude If not satisfied, the patient is to contact, at Vitalité HA, the Patient Safety Quality Department or Risk Management, then the Patient Rep and Comeau (Vitalite); at Horizon HA, the Patient Rep. The regional policy for handling complaints starts with understanding the patient s expectations and Renee- acknowledging receipt of the complaint within 24 hours (max 3-5 business days). Claude.Comeau@vita The Patient Rep approaches the head of the relevant department / responsible manager / program director / local Chief of Staff. litenb.ca The Patient Rep enters the information into a complaints database (currently using several difference databases but looking into a common database system for all types of incidents in NB). The manager / director generally follows up with the patient him/herself with a response required within < 30 days. A group meeting may be held with patient, Patient Rep and manager. An appeals process is in place but it is very seldom used; the case is re-examined and may involve expert opinions from within or outside the province or may go to the HA governing board. The provincial ombudsman is a resource if the appeal is not satisfactory or if the patient chooses this route. Patients can go the legal route if compensation is envisioned (if so, the Patient Rep bows out after meeting with the patient and providing a copy of the chart) case is managed by Risk Management. Similar process for patients who wish to go to the media. Complaints to the Minister of Health are referred to the Patient Rep and then undergo the same process as above. Patient Reps produce quarterly reports for HA boards including # complaints and consultations (i.e., a 4-level system for ranking severity see Appendix D). They observe trends, following up on outliers to explore local issues. Patient Reps connect with QI activities through communication and projects (although this seems to be informal). NS Catherine Gaulton, VP Performance Excellence and General Counsel, Capital District Health Authority; GeneralCounsel.Gaulto n@cdha.nshealth.ca There is no coordinated provincial complaints handling, response or appeals process. Capital Health has a patient experience group plus patient representatives who act as the go to for complaints, sometimes in the first instance, but also for dissatisfaction with the initial handling. There is now centralized access through web and routes. Complaints / compliments are accepted from anyone (government, patients, families, staff, etc.) and are entered into a database where they are coded for follow-up. The database provides for automatic notification of involved clinical teams through their managers and, depending on severity, there is also notification of directors, physician leads, VPs and the CEO. The complaint reports are used to initiate appropriate complaint handling and for quality follow-up. Reports can be generated from the database and are used for team-based quality scorecards and trending. There is no predetermined process for appeals. Matters not sufficiently resolved are elevated by patient representatives internally. Matters and complainants frequently make their way to the CEO and sometimes to the MOH and the Premier's office.? PE Janet Hodder & Joanne Donahoe, Health PEI; jmdonahoe@gov.pe.ca jphodder@gov.pe.ca The single Health PEI system was developed in 2004, modelled on Australia. Providers are encouraged to address complaints locally. Case severity is assessed (perhaps modelled on Australia they re not sure). Beyond this, complaints move to managers and up the chain, ultimately to the Minister of Health or a legal or College route. There is no Ombudsman. The province is moving to an electronic tracking system PSMS (common in Ontario hospitals) for more timely response and data collection for reporting and QI (now 85% complete). No Literature Review - 25

99 Appendix E: Literature Review Report NL NU Name Response Evaluated? Michelle Ryan, Manager Office of Adverse Events, Dept. of Health and Community Services; Joshie Mitsima; & Bessie Joy, Territorial Client Services Coordinator; NL has a system called the Clinical Safety Reporting System (CSRS) to track occurrences in the four RHAs. Initially included was the Datix Complaints module but the four RHAs and the MOH soon realized it needed a lot more work. Eastern Health agreed to continue to work on this Datix Complaints Module for their RHA to try and work out the bugs in it and adapt it to NL. Other RHAs maintained the current systems they had in place. Central Health has an in-house system they developed some time ago. LGH uses a paper system (Excel spreadsheet?). The RHAs do not report aggregated complaint data to the MOH but there will be aggregated provincial reporting on the CSRS. The Eastern Health contact is Heather Predham; Heather.predham@easternhealth.ca. The Central Health contact person is Vanessa Mercer Oldford; Vanessa.MercerOldford@centralhealth.nl.ca. Both CH and EH have complaints staff who have a role in addressing client/patient concerns. On a Ministry level, Bev Griffiths is the Director of Acute Health Services and Emergency Response. They frequently receive complaints/inquiries/requests for information from the public on care/service matters or policy issues. Their role is not individual advocacy to the RHAs but to facilitate a connection between the caller and the RHA to assist them to resolve the issue. NU has a cabinet-approved complaints process including a pamphlet distributed to MLAs and health centres (in several languages), and a process for complaints identified by the Minister of Health and Social Services. Two client service coordinators: one to provide assistance to the people who speak Inuinaqtun (Central or Kitikmeot Region dialect) and another to respond to people who speak Inuktitut (Kivalliq and Qitikmeot) and also to handle any complaints that might come through the Minister s offices or the MLAs No No but a review is planned NT Natasha Brotherston, Manager of Policy; Natasha_Brotherston@gov.nt.ca Main sources of complaints are the two hospitals plus medical travel. Currently all 8 HAs have home-grown patient systems but these are seldom used patients go to the MLA who may be a friend / neighbour / relative. Patient complaints have been higher profile over past year, particularly due to a concern about confidentiality. The Department of Health and Social Services is developing a new system and will shortly seek HA buy-in: 1. Direct approach to provider in health or social services or patient s GP 2. Manager or supervisor 3. Patient Representative (by policy every HA must have someone designated) 4. Lack of resolution moves the case to the CEO of the HA 5. Appeals are possible to the DM of DHSS final step is Minister or MLA who would involve the HA CEO plus the DHSS Physician issues go to a Professional Registrar within the DHSS. The case may be referred to the Complaint Officer (contracted individuals who may be in or outside the territory often lawyer s. For the past 6 months physician issues have been referred to the College of PS in Alberta. No real central tracking of complaints now or link to QI.? YK KD Braden, Director Quality Improvement and Risk Management (QIRM), Yukon Hospital Corporation; KD.Braden@wgh.yk.ca There is a centralized process for managing formal complaints at the Yukon Hospital Corporation (YHC; Yukon s only hospital). Compliment/complaint forms that advise how patients/visitors/family can share their feedback are available in a number of locations in the hospital and at a website link: When a complaint is received the complaint is logged using an on-line incident reporting system and the patient is sent an initial response. Complaints resolution is ideally provided within 30 days after conducting a review with the departments/personnel involved. If a patient is not satisfied with the response they are asked to contact the Director of QI & Risk Management (QIRM) directly to identify any gaps in follow up or further concerns. If they remain unsatisfied they can take their concern forward to the following: CEO of YHC and then onto the Board of Directors for the YHC if needed; Ombudsman and Information and Privacy Commissioners Office for relevant items; Minister of Health or Minister of Parliament. A complaint may be discussed at the YHC Quality Management Council to ensure that systemic improvements are made. The Quality Management Council consists of representatives from YHC that bridge all programs and services with front line representation through to Senior Management Representation. If a complaint is received about the YHC from a government representative they will contact the CEO of the YHC who usually passes this complaint on to the Director of QIRM for resolution. No. Quarterly report to the YHC board; track the implementat ion of improvemen ts based on complaints via review of meeting minutes, etc. Literature Review - 26

100 Appendix E: Literature Review Report 6.0 Compliance with Hospital Accreditation 6.1 ACCREDITATION CANADA Accreditation Canada considers patient complaints and their management in various accreditation standards and guidelines. In particular (the standard is presented, followed by the accompanying guidelines in italics): 7 Under Leadership : o o o 4.3 The organization's leaders engage the community when planning services. Guidelines: The organization's leaders may engage the community and seek input in a variety of formal or informal ways, e.g., focus groups, town hall meetings, feedback or complaint mechanisms, and other community engagement activities and consultation processes. 7.5 The organization's leaders seek input from stakeholders on a regular basis to evaluate the effectiveness of their relationships with them. Guidelines: The frequency of evaluating the effectiveness of relationships with stakeholders should be established and followed by the organization's leaders. They may seek input in a variety of formal or informal ways, e.g., focus groups, feedback or complaint mechanisms, and community engagement activities The organization's leaders identify opportunities for QI based on trends in sentinel events, adverse events and near misses, performance data, patient experience data, and other sources, and develop a plan to prioritize and address those opportunities. Guidelines: Other sources of data used to identify areas for improvement include health outcomes, e.g., changes in the health status of the populations served by the organization; client, community, and staff satisfaction results; complaints; information about the appropriateness of the organization's services; results of utilization reviews, e.g., cost-effectiveness and efficiency; and the level of integration achieved and the continuity of service. Under Service Excellence / Medicine Services : o o 8.7 The team educates clients and families about their rights, and investigates and resolves any claims that these rights have been violated. Guidelines: Client and family rights include the right to have privacy and confidentiality protected, be treated with respect and care, maintain cultural practices, pursue spiritual beliefs, live at risk, and be free from abuse, exploitation, and discrimination. Client rights regarding service delivery include: the right to refuse service or to refuse to have certain people involved in their service; participate in all aspects of their service and make personal choices; have a support person or advocate involved in their service; appeal a service plan decision or file a complaint; take part in or refuse to take part in research or clinical trials; receive safe, competent service; and raise concerns about the quality of service. 8.9 The team responds to client and family complaints in an open, fair and timely way. Guidelines: Clients and families feel comfortable raising concerns or issues. For example, the 7 Personal communication, Wendy Nicklin, CEO Accreditation Canada, April 3, 2012 Literature Review - 27

101 Appendix E: Literature Review Report organization may provide access to a neutral, objective resource person from whom clients and families can seek advice or consultation. o 16.2 The team monitors clients' perspectives on the quality of its medicine services. Guidelines: The team may seek clients' perspectives through surveys, focus groups, interviews or meetings. 6.2 USA JOINT COMMISSION In the USA, the Joint Commission accredits or certifies more than 19,000 health care organizations. A complaint resolution standard requires that accredited hospitals address and resolve complaints from patients and their families (ECRI, 2010). In addition, facilities using Joint Commission accreditation for CMS-deemed status purposes must establish a mechanism for timely referral of patient complaints regarding the quality of their care, or premature discharge to the appropriate CMS-contracted Quality Improvement Organization upon request by a Medicare beneficiary. In addition, it is possible for patients to complain directly to the Joint Commission whose materials state that a source of information is complaints from patients, their families, government agencies, and the public, as well as from an organization s own staff and the media (Joint Commission, 2012). Complainants may provide contact information or remain anonymous. The Joint Commission s process includes evaluating whether the complaint relates to one or more Joint Commission standards. If so, the evaluation will then focus on assessing the organization s overall compliance with those standards. Possible actions: If the complaint raises concerns about a continuing threat to patient safety or if it suggests a failure to comply with Joint Commission standards, an unannounced or unscheduled on-site evaluation of the organization may be organized. A written response to the complaint may be requested. If the organization is scheduled for its accreditation survey in the near future, the complaint and the organization s compliance with related standards may be reviewed at that time. The complaint details may be integrated into the Joint Commission database that monitors quality issues and tracks trends. Literature Review - 28

102 Appendix E: Literature Review Report 7.0 Discussion 7.1 LITERATURE FROM OTHER COUNTRIES A review of the literature on patient complaint systems in health care systems beyond Canada s borders is instructive. Published information for seven countries outside Canada was located, i.e., Australia, Finland, Ireland, the Netherlands, NZ, the UK and the USA. In addition, from examination of standards set by Accreditation Canada and the USA Joint Commission, it is clear that health care accreditation is a major driver in pushing systems to learn from patient complaints in order to improve care. Common themes and ideas emerged from the literature review of systems in other countries: Patients should understand how to provide complaints and compliments about their care, to both ensure optimal care for themselves and their families, and to help the system to improve. Direction may come from information prominently displayed in care venues (e.g., posters, websites) and / or verbal / written instruction to patients when they access care, e.g., when admitted to hospital. Access to complaints processes should be very clear and user-friendly. This may involve the assistance of a patient advocate / ombudsman to help a patient / family navigate the system. Early on, attention should be paid to acknowledging and clearly understanding the details and nature of a patient s complaint and motivation for complaining; this can lead to managing patients expectations of the system. Some complaints may not be appropriate and processes should be developed to deal with these at an early stage to prevent them from bogging the system down. To as great an extent as possible, complaints should be dealt with at a local level. If an informal local process does not satisfy the complaint, or is not feasible (e.g., patient is not comfortable, local system is weak or unresponsive, complaint is of a serious nature), the service / facility should have a clearly identified resource for a patient / family / advocate) to approach. If this second stage process does not satisfy the complaint, or is not feasible, an independent process (can be multistaged) should be available to carry complaints to some type of resolution. Comprehensive data should be collected for regular reporting purposes and also for trending. Complaints systems should be linked to QI activities to ensure system weaknesses are identified and gaps filled. A very useful Danish report was located that reviewed systems in nine countries including Canada (Lister et al., 2008). 8 The report distilled the authors observations about commonalities according to 14 themes (Table 3). NOTE: A number of countries have developed single stream patient complaint systems where ALL complaints are managed by the same processes, at least early on. This includes complaints focussed on care providers (particularly physicians). These models cannot be extrapolated to Canada where regulatory colleges deal with complaints against their members. 8 Presumably a number of other countries have complaint management systems in place or under development but our search did not identify published information about these systems. Literature Review - 29

103 Organizational responsibilities Legislation and regulation Objectives Appendix E: Literature Review Report Definitions of complaints Processes must be simple Short timescales Staffing Numbers of complaints Difficult to initiate complaints Attitudes of staff (fear of blame) Apology Redress Monitoring Learning from complaints Improvement suggestions TABLE 3: LISTER ET AL. (2008) SUMMARY OBSERVATIONS ACROSS NINE REVIEWED JURISDICTIONS (adapted) Resolve complaints informally with the care team as possible. Ideally, complaint support staff members train front line staff about how to deal with complaints. The complaint appeals organization must be independent with the most important functions being an appeals processes and follow-up actions, particularly where complaints raised system-wide issues. Ensure these (a) provide a basis for patient/client rights and staff expectations (standards or guidelines); (b) ensure that providers and users know what is reasonable, what must be improved and when complaint is justified; and (c) outline a clear simple process. Ensure that objectives translate the ideals and values of service providers and complaints agencies into performance measures, encouraging feedback from staff and users to improve services. Ensure that definitions are broad but measurable, including severity assessment. Handling complaints alongside adverse events and improvement suggestions and complaints from staff provides a common basis for learning and improvement. Develop a system of local resolution (informal and formal) supported by independent review processes undertaken by a second agency. Avoid handoffs between agencies. Ensure that informal resolution and apology is immediate where possible (within 4-5 weeks). Timescales beyond this should be short but related to the complexity of the issue. Where policy issues are raised, separate longer term review may be required. Ensure the right staff members are in place. Most directly involved in local complaints are front line care teams supported by a local complaints team. Both complainants and people who are the subject of complaints should have access to independent support staff or volunteers to help them. Special arrangements are required for vulnerable groups, e.g., children in care and people with mental health problems. Understand that numbers do not reflect opinions of services. Where a positive attitude was taken numbers increased. It was also apparent that some complaints do not reflect a realistic understanding of what patients should reasonably expect and was seen as important to be able to close such complaints without overloading the system. Understand that it can be difficult to initiate complaints. Obstacles include: perception that the process would be long and complex, perception that no action would be taken anyway, and /or fear that there might be repercussions. People most often want an explanation, an apology and reassurance that the same thing would not happen to others. Patient / client based organisations can help. Understand that attitudes are the most important barriers to positive approaches to complaints. Required are a supportive management culture and greater clarity as to what can reasonably be expected. Understand that apology remains difficult for staff. It requires that apology and resolution of complaints should be separated from acceptance of liability and blame as much as possible. Understand that no fault compensation schemes are separate from complaints systems, but small gifts recognize costs and inconvenience and help recover confidence, particularly when accompanied by explanation and apology, e.g., one complaints team accompany a personal apology and explanation with flowers to thank the complainant for helping them improve the system. Ensure that monitoring occurs at local and national levels and involves real life examples rather than just statistics. Also, de-emphasize time scales to the exclusion of other outcomes. A national annual report on complaints and QI is helpful. Understand that learning should occur at local and national levels. Best practice at the local level saw complaints support staff working closely with QI teams. Where national complaint support agencies do not have a broad remit they need strong links to QI and standard-setting agencies. Ensure there is a system for suggestions to increase the responsiveness and accessibility of complaints systems and their efficiency and timeliness. In the Lister et al review, comments included calls for a positive approach to complaints, increased transparency, and keeping complainants informed of progress. Suggested were sharing of experience through an international conference and on-line community. Literature Review - 30

104 Appendix E: Literature Review Report Notwithstanding broad multi-jurisdictional interest in and/or commitment to complaints management systems it is worth noting the literature is largely descriptive. Even within the descriptive literature, only a few jurisdictions (Australia, NZ and the UK) stand out for having fully documented their complaint systems, including their evolution over time. Although complaint management systems are widely seen as mechanisms for restoring trust and deescalating conflict, there is very little empirical evidence regarding best practice or the overall contribution of complaints management systems within the health sector. There is significant emphasis in the literature on the potential link between complaints management and QI. However, in most instances the linkage remains highly ephemeral. Australia and NZ stand out in that they require all formal complaints to be entered into an incident information management system and, in the case of clinical incidents, to be cross referenced to a clinical incident form. There also appears to be an effort to reinforce a culture of reporting and responsibility in the health sector with patient concerns and complaints recast as consumer reported incidents thereby suggesting they have the same value as traditional staff reported incidents. (ACSQHC, 2004/2012) Australia, NZ and the UK also stand out for their utilization of complaint severity scoring systems. There are possible explanations, besides technical issues such as incident capture and severity scoring systems, for the apparent challenges in implementing a QI focus. Many complaints may offer little in the way of substantive QI opportunities, e.g., the most recent changes in the UK system have served to re-emphasize local and informal complaint resolution, a move reflective of the routine and minor nature of most issues. Also worth noting is the presence of complaints which are potentially substantive but relatively intractable in nature. For example, the most UK common complaint year after year relates to access and waiting. Assuming a complainant s wait for service has been managed in a manner consistent with other patients then the complaint is about a structural issue and the default focus for the complaint system must revert to consumer satisfaction as opposed to QI, per se. The emphasis on local resolution is also worthy of attention. Data are scarce but statistics from the UK indicate the NHS received 133,400 written complaints in 2006/2007, 94% of which were resolved or went no further than the local level. Perhaps more interesting is the fact that related research found 14% of patients are in some way dissatisfied with the NHS but 95% of them fail to complain (National Audit Office, 2008). To the degree that the UK is representative, an emphasis on local resolution seems to be effective, but overall the complaint management system may cater to a minority of dissatisfied patients and perhaps at significant cost. The estimated cost of handling complaints in the NHS in 2006/2007 was 89 million. The failure to complain is not unique to the UK as others have documented a very modest correlation between documented adverse events and complaints (Bismark, 2006a). The failure to complain has been linked to lack of awareness of options/avenues but also to socio-demographic considerations such as age, ethnicity and income. Although the best documented and seemingly most mature complaints management systems provide avenues for appeal, the number of avenues available in the UK has been reduced and the willingness of appeal bodies in the UK, Australia and NZ to actually hear appeals is limited. In 2010/2011, the UK PHSO Literature Review - 31

105 Appendix E: Literature Review Report received 15,066 complaints but only agreed to formally investigate 351. A further 487 complaints were rapidly resolved by direct intervention on the part of the PHSO, i.e., no investigation was required, but the majority of complaints (14,023) were returned to the local level for further action, dismissed as unsubstantiated or addressed, or withdrawn (PHSO, 2011b). There is some recognition that the presence of appeal bodies has potential to reduce local resolution efforts; at least 50% of appeals in the UK are routinely returned to the local level for additional work. As noted earlier, some jurisdictions have endeavored to create complaint management systems that have a broad mandate to manage all health sector complaints. Notable in this regard is the UK where significant fragmentation existed until Despite significant changes, fragmentation remains an issue and there is concern that the complaints management system is not keeping pace with structural reforms. For example, the NHS is increasingly contracting out to private providers who are not legislatively bound to replicate the complaints management process. Even in instances where such providers do so, there is no avenue to appeal decisions as the PHSO has no jurisdiction. System fragmentation is an under-appreciated reality in most patient complaints management processes and is certainly true in Canada. Even in the most advanced Canadian jurisdiction a consumer pursuing a complaint could find themselves engaged in multiple processes (health authority, professional colleges, etc.), each with different requirements, timelines and appeal processes. 7.2 FINDINGS FROM WITHIN CANADA A key aspect of this project was determining what the patient complaint systems look like in Canada s 12 other P/Ts. These systems have evolved independently with similarities and differences among them, in part related to the size and sophistication of the jurisdiction. Varying models of regionalization have also led to differences, e.g., Ontario is not regionalized (aside from Local Health Integration Networks [LHINs]) and two provinces have only one HA (PEI and Alberta) the systems in such jurisdictions make them fundamentally different. All systems encourage local resolution of concerns and have subsequent processes, although these vary. Also, all jurisdictions do not really venture into management of complaints concerning competence of providers as these fall to regulatory colleges. Several models of complaint systems appear to be in use: Unique solutions: o NWT: Each of eight HAs has a homegrown process that is seldom used; patients tend to go directly to their Member of the Legislative Assembly (MLA) who, due to the small communities involved, may be a friend, neighbour or relative. The Department of Health & Social Services (DHSS) is working to develop a consistent process that will protect confidentiality as this is seen to be at risk with the rapid movement to a more political arena. Their ideal model will move from attempts at local resolution to a manager / supervisor and/or a HA Patient Representative (required in every HA). Lack of resolution will move the case to the CEO of the HA. Appeals are envisioned at the DM level with a final step being the Minister or MLA. Complaints currently Literature Review - 32

106 Appendix E: Literature Review Report originate from care at one of the two hospitals but also from issues with reimbursement and coverage for medical travel within the territory or to Edmonton. o Nunavut: With its vast geography, tiny population, remote location and culture / language issues, Nunavut is unique. Two central DHSS Client Service Coordinators provide assistance in several Arctic languages. Complaints move quickly to a political venue with MLAs involved early. System confined to a hospital: o o Ontario: Ontario has not regionalized health care aside from Local Health Integration Networks (LHINs). Legislation (2010) requires hospitals to survey patients annually and to develop transparent patient relations processes. Examination of a very large academic consortium in Toronto reveals a Patient Relations Office / Hospital Ombudsman that handles patient complaints, provides consultation and education for staff, and advocates for system change. Their annual report displays number of complaints etc. and their ultimate disposition but there is no indication of a secondary procedure or appeal route. Unlike some other jurisdictions, the Ontario ombudsman does not have the authority to examine hospital affairs and this has recently received media attention. Yukon: The patient complaints system revolves around the territory s only hospital in Whitehorse. Failure to resolve problems locally can lead to involvement of the hospital CEO or board, the Yukon ombudsman or politicians. System contained within a HA: o o o Manitoba and Nova Scotia: There are no coordinated provincial complaints handling, response or appeals processes but well-established hierarchical complaints systems are in place in individual HAs, particularly the larger ones like Winnipeg and Capital Health (Halifax). Newfoundland & Labrador: Systems are specific to each of the four HAs. Department of Health staff members frequently receive complaints, inquiries and requests for information from the public. Their role is to facilitate a connection between the caller and the RHA to assist in issue resolution, not to get involved in individual cases. An appeals process was not evident. New Brunswick: The two HAs have their own processes. Failure of local resolution moves a complaint to a HA Patient Representative who approaches the head of the relevant department, responsible manager, program director, etc. and these personnel take over the interaction with the patient. A seldom-used appeals process involves re-examining the case, often with expert opinions from within or outside the province. Appeals may go to the HA governing board. The provincial ombudsman is a resource if the patient chooses this route. Of note, New Brunswick was the only jurisdiction to describe use of a severity classification system (Appendix D). Literature Review - 33

107 Appendix E: Literature Review Report A single HA / health corporation in the province means the HA / provincial systems are the same: o o PEI: A single health corporation (Health PEI) operates and delivers publicly funded health services on the Island. Failure of local resolution moves issues to a manager and up the chain, ultimately to the Minister of Health or a legal or College route; there is no Ombudsman. Alberta: With the creation of Alberta Health Services (AHS) in 2008 from nine HAs, development of a single HA / provincial patient concerns system has been evolving. Failure of local resolution moves complaints to the unit manager, director, and VP levels, as necessary. Central intake at the AHS Office of Patient Concerns is an option if patients choose not to take the local route or if the issue is not resolved. Patient Concerns Consultants gather information from all parties and clarify what is outstanding. Lack of resolution at this stage moves the process to the AHS Director of Patient Concerns and beyond this to the Alberta Ombudsman whose office took on health care concerns in With a focus on administrative fairness, the Ombudsman s Office will only take cases after all AHS avenues have been completed. Local and HA systems move up to a Provincial system: o o Quebec: The local Complaints and Service Quality Commissioner of the facility delivering services is the first step; a local Users Committee can act as a resource as can CAAPs (regional community organizations that assist and support users). Second recourse is via the Quebec Ombudsman who accepts cases where the complainant is dissatisfied with the conclusions of the Complaints Commissioner or a response from the Commissioner has not been received within 45 days. Private facilities and complaints related to physicians, dentists and pharmacists are not accepted. These cases move from the local Commissioner to a medical examiner system that also has a back-up review procedure. Saskatchewan: A provincial system has been in place since Beyond attempts at local resolution, complaints go to an RHA Client Rep / Quality Care Coordinator, who learns the story, helps the patient navigate the system, investigates, and suggests or enacts solutions. If patient is not satisfied with this outcome, an appeal at the RHA level is possible. The Provincial Ombudsman can be approached, although the scope of cases taken is narrow. There are two Provincial Quality of Care Coordinators at Saskatchewan Health who handle the small number of complaints not settled in the RHAs and not suitable for the Ombudsman. Exceptions may go to litigation and a few cases are never resolved in a satisfactory manner. Beyond descriptions of the overall structure and various steps in P/T patient complaints systems, interviews touched on issues such as capture of complaints information in databases, use of a severity scoring system, reporting of results, links between patient complaints and QI activities, and system evaluations. There are generally early efforts in these areas. The more informal and less centralized the system, the less likely is there to be a common data capture and analysis system, and without this, meaningful reporting and linkages to QI are not possible. Literature Review - 34

108 Appendix E: Literature Review Report However, with its fairly sophisticated system, Saskatchewan has moved ahead in a number of areas and will be presented here as an example. With respect to data capture, reporting and linkage to QI, RHAs enter complaints data into a database classifying complaints according to specified categories, program area, outcome/action, response to resolution, time to resolution, etc. Aggregate information is viewed by the MOH. An annual report is shared with MOH program areas looking at types of concerns, trending, and specific initiatives (e.g., suicides while in care and falls). RHAs also report routinely to their boards. Provincial coordination of workers in this field are also led by the MOH including twice annual meetings with Patient Care Coordinators with speakers / networking, a listserv that shares RHA Coordinators experience, and regular contact among RHA Coordinators including the MOH. With respect to evaluations, it was not clear than any Canadian jurisdictions have conducted formal evaluations of their patient complaints systems, although several are looking forward to doing so. Interviewees were generally keen to learn from BC s experience. 8.0 Conclusions Complaint management systems have long existed at the point of service in health care organizations. Escalation processes have also existed. More formal, statute-based systems / processes are a more recent phenomenon, existing in some jurisdictions since the 1990s but still absent in others, including some Canadian jurisdictions. Systems for managing complaints vary in form but all appear to be multi-stage in nature, with initial emphasis on local, relatively informal, resolution. There is some evidence to suggest these local options have been effective in most cases. Local point-of-service options are typically supplemented by one or two more formal options with the final option being an independent or quasi-independent review of the complaint, although the latter may be very limited with respect to types of cases examined. The published literature regarding complaints management tends to be implicitly or explicitly positive but this review found little in the way of empirical research regarding best practice or the overall impact of formal statute-based initiatives in the health sector. To the extent that evaluative literature exists, it appears to be predominantly from Australia, NZ and the UK. There is evidence to suggest complaint systems capture a minority of complaints and that the most formal structures serve an even smaller subset of the population. Awareness and socio-demographic factors are identified barriers to the use of complaint management systems. Challenges linking complaints to quality improvement processes are also evident, as is fragmentation in the complaint process, i.e., the need for patients to deal with more than one process. These challenges persist to varying degrees, even in fourth generation complaint management systems. Literature Review - 35

109 Appendix E: Literature Review Report 9.0 References Alhafaji FY, Frederiks BJ, Legemaate J. The Dutch system of handling complaints in health care. Med Law 2009 Mar;28(2): Australian Commission on Safety and Quality in Health Care (ACSQHC). Windows into Safety and Quality in Health Care Sydney: ACSQHC, 2009:1-98. Available at: Australian Commission on Safety & Quality in Health Care (ACSQHC). Patient-centred care: improving quality and safety by focusing care on patients and consumers Sept. Available at: Australian Commission on Safety & Quality in Health Care (ACSQHC). Better practice guidelines on complaints management for health care services / Available at: Australian Council for Safety and Quality in Health Care. Complaints management handbook for health care services July. Available at: complntmgmthbk.pdf Parliamentary and Health Service Ombudsman (PHSO). Background to the policy and practice of the Health Service Ombudsman for England on sharing and publishing information about NHS complaints. London: Parliamentary and Health Service Ombudsman; Available at: data/assets/pdf_file/0012/6303/phso-0130-sharing-info-policy-background- 5.pdf Birmingham J. Notifying patients of their right to complain. Hosp Case Manag 2011 Jun;19(6):93-4. Bismark MM, Brennan TA, Paterson RJ, Davis PB, Studdert DM. Relationship between complaints and quality of care in New Zealand: a descriptive analysis of complainants and non-complainants following adverse events. Qual Saf Health Care 2006a;15(1): Available at: Bismark M, Dauer E, Paterson R, Studdert D. Accountability sought by patients following adverse events from medical care: the New Zealand experience. CMAJ 2006b;175(8): Available at: Bismark MM, Studdert DM. Realising the research power of complaints data. N Z Med J 2010 May 14;123(1314):12-7. Available at: Bismark MM, Spittal MJ, Gogos AJ, Gruen RL, Studdert DM. Remedies sought and obtained in healthcare complaints. BMJ Qual Saf Sep;20(9): Carers Federation Ltd. Independent Complaints Advocacy Service Available at: Christiaans-Dingelhoff I, Smits M, Zwaan L, Lubberding S, van der Wal G, Wagner C. To what extent are adverse events found in patient records reported by patients and healthcare professionals via complaints, claims and incident reports? BMC Health Serv Res Feb 28;11:49. Available at: Clement M, Gagnon E. The Quebec complaint examination system: stakeholder perspectives on the purpose and intake of complaints. Health Care Anal 2006 Mar;14(1): Condon D. Is anyone listening to your complaints? Irish Health Available at: Department of Health (DOH). Listening, responding, improving: A guide to better customer care Available at: Literature Review - 36

110 Appendix E: Literature Review Report Department of Health and Children (DHC), Ireland. Building a culture of patient safety: Report of the Commission on Patient Safety and Quality Assurance July. Available at: Department of Health, Social Services and Public Safety of Northern Ireland (DHSSPSNI). Complaints in health and social care: standards & guidelines for resolution & learning. [Belfast (Northern Ireland)]. 2009a. Available at: Department of Health, Social Services and Public Safety of Northern Ireland (DHSSPSNI). Guidance on complaints handling in regulated establishments and agencies. [Belfast]: Northern Ireland. 2009b. Available at: ECRI. Managing patient grievances and complaints. Plymouth Meeting (PA): ECRI Institute; Available at: Gulland J. Second-tier reviews of complaints in health and social care. Health Soc Care Community May;14(3): Health & Disability Commissioner (HDC) Statement of intent Auckland (NZ): Office of the Health and Disability Commissioner; Available at: Health & Disability Commissioner Statement of intent "What we plan to do from 2011 to 2014": Full easy read version. Auckland (NZ): Office of the Health and Disability Commissioner; Available at: Health Quality Council of Alberta (HQCA). Patient concerns/complaints resolution: provincial framework. Calgary (AB): Health Quality Council of Alberta (HQCA); Available at: Hsieh SY. Healthcare complaints handling systems: a comparison between Britain, Australia and Taiwan. Health Serv Manage Res 2011 May;24(2):91-5. Irish Statutes Book. S.I. No. 652/2006-Health Act 2004 (Complaints) Regulations Available at: Joint Commission. Report a complaint Available at: Lister G, Rosleff F, Boudioni M, Dekkers F, Jakubowski E, Favelle H. Handling complaints in health and social care: international lessons for England. London: National Audit Office (NAO); 2008.Available at: Managing patient complaints in Ireland: people matter [presentation]. [Cambridge (UK)]: University of Cambridge. Judge Business School; Available at: Manning J. Access to justice for New Zealand health consumers. J Law Med 2010 Sep;18(1): Available at: McAuliffe E. Report on the Consultation on the Health Act, 2004 (Part 9) - complaints. Dublin (Ireland): Prepared for the Department of Health and Children; Available at: McSherry B, Klauber S. Who do I turn to? Resolving complaints by mental health consumers and carers. J Law Med 2011 Jun;18(4): Medicare Part D: complaint rates are declining, but operational and oversight challenges remain. Washington (DC): US Government Accountability Office (GAO); Available at: Medical Protection Society (MPS). NHS complaints procedure. London: Medical Protection Society (MPS). Professional Support and Expert Advice; Available at: Natangelo R. Patient agencies and complaints in Italy. Qual Saf Health Care 2006 Jun;15(3):223. Literature Review - 37

111 Appendix E: Literature Review Report National Audit Office (NAO). Feeding back? Learning from complaints handling in health and social care. London: The Comptroller and Auditor General; Available at: National Health Service (NHS). The handbook to the NHS constitution for England Available at: Parliamentary and Health Services Ombudsman (PHSO). Sharing and publishing information about NHS complaints: the policy and practice of the Health Service Ombudsman for England. London: Parliamentary and Health Service Ombudsman; Available at: data/assets/pdf_file/0011/6302/phso Sharing-info-policy-and-practice-5a.pdf Parliamentary and Health Services Ombudsman (PHSO). Responsive and accountable? The Ombudsman's review of complaint handling by government departments and public bodies London: Parliamentary and Health Service Ombudsman; 2011a. Available at: Parliamentary and Health Service Ombudsman (PHSO). Listening and Learning: the Ombudsman s review of complaint handling by the NHS in England London: Parliamentary and Health Service Ombudsman; 2011b. Available at: data/assets/pdf_file/0019/12286/listening-and-learning-screen.pdf Parliamentary and Health Services Ombudsman (PHSO). One year corporate business plan London: The Parliamentary and Health Service Ombudsman; 2011c. Available at: data/assets/pdf_file/0020/9830/0154-business-plan-artwork pdf Parliamentary and Health Services Ombudsman (PHSO). Four year strategic plan London: The Parliamentary and Health Service Ombudsman; 2011d. Available at: data/assets/pdf_file/0003/9831/0153-strategic-plan-artwork pdf Parliamentary and Health Services Ombudsman (PHSO). What we can help with Available at: Pelling M, Spath P. New CMS guidelines for managing complaints. Hosp Peer Rev 2006 Jan;31(1):5-6. Rogers S, Vilhena B. Benchmarking patient complaints data across Ontario: University Health Network invited collaboration. Healthcare Quarterly Available at: Temelkovski S, Callaghan K. Opportunities to learn from medical incidents: a review of published reports from the Health and Disability Commissioner. N Z Med J 2010 May 14;123(1314): Thacker EL. The United States long-term care ombudsman program: Understanding the real-life circumstances underlying resident complaint data. Expert Review of Pharmacoeconomics and Outcomes Research 2006;6(6): Available at: Troyer JL, Sause WL. Complaints against nursing homes: comparing two sources of complaint information and predictors of complaints. Gerontologist 2011 Aug;51(4): Valimaki M, Kuosmanen L, Karkkainen J, Kjervik DK. Patients' rights to complain in Finnish psychiatric care: an overview. Int J Law Psychiatry 2009 May;32(3): Veterans Health Administration (VHA). VHA Patient Advocacy Program Available at: Veterans Health Administration (VHA). VHA clinical appeals Available at: Wendel S, de Jong JD, Curfs EC. Consumer evaluation of complaint handling in the Dutch health insurance market. BMC Health Services Research 2011;11:310. Available at: Wise J. Handling complaints poorly makes problems worse, says NHS ombudsman. BMJ 2010;341:c5874. Your rights when using a health or disability service in New Zealand and how to make a complaint. Auckland (NZ): Health and Disability Commissioner; Available at: Literature Review - 38

112 Appendix E: Literature Review Report Appendix A: Complaint severity rating scale described in Australia The Complaints Management Handbook for Health Care Services (Australian Council, 2005) suggests use of a severity rating scale to identify, assess, understand and manage risk. Presented is a matrix of the severity of the consequences of a complaint versus the likelihood such a situation will recur (Table 5). The Handbook suggests that use of such a system can provide consistency and reliability over time. One purpose is to identify high risk complaints that raise significant safety, legal or regulatory issues to take to senior management immediately. TABLE 5: SUGGESTED COMPLAINTS SEVERITY RATING SCALE The Handbook goes on to detail how the consequences of an incident can be defined as serious, major, moderate, minor or minimum by reference to factors such as: Level of physical injury to a patient Failure of administrative systems for patient care, e.g., loss of test results Level of injury to staff (for occupation health and safety incidents) Level of financial or corporate losses (including litigation and adverse publicity) Level of harm to the customer service relationship Examples of severity classification (Australian Council, 2005; page 42): SERIOUS INCIDENT A patient has died as a result of receiving health care in a manner that is unrelated to the natural course of the illness and differing from the expected outcome of patient management. Death of a staff member or visitor. Complete loss of service capability. Huge financial loss. Serious threat to customer service relationships, permanent harm to reputation of the service. MAJOR INCIDENT A patient has suffered harm as a result of receiving health care in a manner that is unrelated to the natural course of the illness and differing from the expected outcome of patient management, resulting in hospitalization (or increased length of stay). Permanent injury to staff members or visitors. Loss of service capability including cancelled appointments. Major financial loss. Serious breakdown of customer service relationships. Literature Review - 39

113 Appendix E: Literature Review Report Appendix B: Complaint severity rating scale described in the UK Listening, Responding, Improving: A Guide to Better Customer Care (DOH, 2009) describes a three-step process used to gauge the impact of complaints on the people involved, the potential risks to the organization and the response required. It is suggested that complaints are categorized when they are first received with the categories reviewed based on subsequent investigations. STEP 1 Decide how serious the issue is: LOW: o Unsatisfactory service or experience not directly related to care with no impact or risk to provision of care OR unsatisfactory service or experience related to care, usually a single resolvable issue. o Minimal impact and relative minimal risk to care or service provision. o No real risk of litigation. MEDIUM: o Service or experience below reasonable expectations in several ways, but not causing lasting problems. o Has potential to impact on service provision. o Some potential for litigation. HIGH: o Significant issues regarding standards, quality of care and safeguarding of or denial of rights. o Complaints with clear quality assurance or risk management issues that may cause lasting problems for the organization and so require investigation. o Possibility of litigation and adverse local publicity OR serious issues that may cause long-term damage such as grossly substandard care, professional misconduct or death. o Will require immediate and in-depth investigation. o May involve serious safety issues. o A high probability of litigation and strong possibility of adverse national publicity. STEP 2 Decide how likely the issue is to recur (rare, unlikely, possible, likely, almost certain) STEP 3 Categorize the risk (low, moderate, high, extreme) (Table 6) TABLE 6: COMPLAINTS SEVERITY MATRIX Literature Review - 40