Associate Professor David Greenfield

Key findings from studies into health service accreditation programs Australian Institute of Health Innovation Associate Professor David Greenfield Sydney, Australia 10 September 2014

Presentation on behalf of the UNSW ACCREDIT Team: Prof Jeffrey Braithwaite Prof Johanna Westbrook Dr Anne Hogden Dr Virginia Mumford Ms Deborah Debono

Types of regulation

How widespread is healthcare accreditation? Accreditation of health organisations is practised in more than 70 countries. 22 national bodies. One international organisation: ISQua.

What the critics say... Healthcare clinical and organisational performance Accreditation survey Time

What the advocates claim... Accreditation survey Healthcare clinical and organisational performance Time

ACCREDIT project partners

Research aims and studies Study 1. Accreditation models Study 2. Critical elements of accreditation Study 3. Standards and their impact Study 12. Effective and ineffective use of accreditation processes Study 11. Surveyors and their place in accreditation Study10. Unannounced surveys Does accreditation make a difference to quality and performance? What are its cost implications and what benefits are realised? How can it be improved? To what extent can consumer involvement be improved? Study 4. Standards: consumer participation and IT Study 5. The patient experience Study 6. Costbenefit analysis of accreditation Study 9. Tracer methodology Study 8. Public disclosure of accreditation results Study 7. A natural experiment of what ACSQHC does to transform accreditation

The benefits of accreditation: Having a positive accreditation result is associated with good organisational and clinical performance. References: Braithwaite, J., Greenfield, D., Westbrook, J., Pawsey, M., Westbrook, M., Gibberd, R., Naylor, J., Nathan, S., Robinson, M., Runciman, B., Jackson, M., Travaglia, J., Johnston, B., Yen, D., McDonald, H., Low, L., Redman, S., Johnson, B., Corbett, A., Hennessy, D., Clark, J. and Lancaster, J. (2010) Health service accreditation as a predictor of clinical and organizational performance: a blinded, random, stratified study, Quality and Safety in Health Care, 19(1): 14-21. Greenfield, D., Pawsey, M. and Braithwaite. J. (2012) The role and impact of accreditation on the healthcare revolution [O papel e o impacto da acreditação na revolução da atenção à saúde], Acreditação, 1(2): 64-77. Greenfield D., Pawsey, M. and Braithwaite. J. (2013) Accreditation: a global regulatory mechanism to promote quality and safety, in Sollecito, W. and Johnson, J. (eds), Continuous Quality Improvement in Health Care, 4 th edition, New York, Jones and Barlett Learning, pp. 513-531. Greenfield, D., Kellner, A., Townsend, K., Wilkinson, A. and Lawrence, S.A. (2014) Health service accreditation reinforces a mindset of high performance human resource management: lessons from an Australian study, International Journal for Quality in Health Care, accepted 17/03/14.

The benefits of accreditation: Accreditation promotes positive quality and safety cultures across organisational boundaries. References: Greenfield, D., Pawsey, M. and Braithwaite, J. (2011) What motivates health professionals to engage in the accreditation of healthcare organisations? International Journal for Quality in Health Care, 23(1):8-14.

Percentage Question 23 In my opinion accreditation has improved the way my organisation cares for patients/residents. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 24: In my opinion accreditation has improved the organisational processes and systems of my organisation. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 70% 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 25: In my opinion accreditation has improved the management of my organisation. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 15: In my opinion accreditation is important to healthcare staff. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 19: In my opinion the accreditation program is well regarded by staff in my organisation. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

The benefits of accreditation: Accreditation can be used to create and build quality and safety improvements. References: Greenfield D., Pawsey, M. and Braithwaite. J. (2013) Accreditation: a global regulatory mechanism to promote quality and safety, in Sollecito, W. and Johnson, J. (eds), Continuous Quality Improvement in Health Care, 4 th edition, New York, Jones and Barlett Learning, pp. 513-531. Greenfield, D., Pawsey, M. and Braithwaite. J. (2012) The role and impact of accreditation on the healthcare revolution [O papel e o impacto da acreditação na revolução da atenção à saúde], Acreditação, 1(2): 64-77.

Percentage Question 1: In my opinion accreditation is necessary to promote patient/resident safety within healthcare organisations. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 60% 50% 40% 30% 20% 10% ALL AGPAL ACSAA ACHS 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Responses

Percentage Question 2: In my opinion accreditation is necessary to promote quality care within healthcare organisations. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Responses

Percentage Question 3: In my opinion continuous improvement is an important element of the accreditation program. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 70% 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 26: In my opinion accreditation has improved patient/resident satisfaction with my organisation. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

The benefits of accreditation: Short notice surveys (SNS), or unannounced surveys, offer useful options for accreditation schemes. References: Greenfield, D., Moldovan, M., Westbrook, M., Jones, D., Low, L., Johnston, B., Clark, S., Banks, M., Pawsey, M., Hinchcliff, R., Westbrook, J. and Braithwaite, J. (2012) An empirical test of short notice surveys in two accreditation programs, International Journal for Quality in Health Care, 24(1): 65-71.

Percentage Question 40: In my opinion providing organisations with only short notice prior to survey visits would improve accreditation programs. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 32: In my opinion the following elements of the accreditation program drive changes in my organisation: survey visits. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Responses

Percentage Question 36: In my opinion surveys require too much documentary evidence. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 40% 30% 20% 10% ALL AGPAL ACSAA ACHS 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

The benefits of accreditation: The patient journey survey (PJS) method in the accreditation process is a valuable approach. References: Greenfield, D., Hinchcliff, R., Westbrook, M., Jones, D., Low, L., Johnston, B., Banks, M., Pawsey, M., Moldovan, M., Westbrook, J. and Braithwaite, J. (2012) An empirical test of accreditation patient journey surveys: randomised trial, International Journal for Quality in Health Care, 24(5): 495-500.

Percentage 60% Question 41: In my opinion following patient experiences of moving through healthcare organisations would improve surveys. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

The benefits of accreditation: Accreditation survey and surveyor reliability can be enhanced through a well facilitated program, collaborative stakeholder relationships and an experienced surveyor workforce. References: Greenfield, D., Pawsey, M., Naylor, J. and Braithwaite, J. (2009) Are healthcare accreditation surveys reliable?, International Journal of Health Care Quality Assurance, 22(2): 105-116. Greenfield, D., Pawsey, M., Naylor, J. and Braithwaite. J. (2013) Researching the reliability of accreditation survey teams: lessons learnt when things went awry, Health Information Management Journal, 42(1): 4-10. Lancaster, J., Braithwaite, J. and Greenfield, D. (2010) Benefits of participating in accreditation surveying, International Journal of Health Care Quality Assurance, 23(2): 141-152. Greenfield, D., Braithwaite, J. and Pawsey, M. (2008) Health care accreditation surveyor style typology, International Journal of Health Care Quality Assurance, 21(5): 435-443.

Percentage Question 43: In my opinion surveyors/assessors have the appropriate knowledge and skills. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 48: In my opinion individual surveyors/assessors are consistent in their judgements. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 49: In my opinion survey/assessor teams are consistent in their judgements. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

The benefits of accreditation: The empirical evidence base for accreditation programs and the development of accreditation standards have not been compelling in the past but these are improving. References: Greenfield, D. and Braithwaite, J. (2008) Health sector accreditation research: a systematic review, International Journal for Quality in Health Care, 20(3): 172-183. Hinchcliff, R., Greenfield, D., Moldovan, M., Pawsey, M., Mumford, V., Westbrook, J. and Braithwaite, J. (2012) Narrative synthesis of health service accreditation literature, BMJ Quality and Safety, 21(12): 979-991. Greenfield, D., Pawsey, M., Hinchcliff, R., Moldovan, M. and Braithwaite. J. (2012) The standard of healthcare accreditation standards: a review of empirical research underpinning their development and impact, BMC Health Services Research, 12(1):329. Greenfield, D., Pawsey, M. and Braithwaite. J. (2012) A peer-to-peer model to improve patient safety: harnessing accreditation programs, American Journal of Medical Quality, 27(4): 353-354. Greenfield, D. and Braithwaite, J. (2009) Developing the evidence base for accreditation of healthcare organisations: a call for transparency and innovation, Quality and Safety in Health Care, 18(3): 162-163.

The benefits of accreditation: Economic evaluation of accreditation programs using cost-benefit analysis is at a rudimentary stage, but most quality and safety initiatives have not been rigorously subject to cost-benefit analyses. References: Mumford, V., Forde, K., Greenfield, D., Hinchcliff, R. and Braithwaite, J. (2013) Health services accreditation: what is the evidence that the benefits justify the costs? International Journal for Quality in Health Care, 25(5): 606-620.

Percentage Question 8: In my opinion accreditation does not require financial costs beyond what is normally invested in quality and safety activities in my organisation. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 40% 30% 20% 10% ALL AGPAL ACSAA ACHS 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 6: In my opinion organisations should be given financial incentives to participate in an accreditation program. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 9: In my opinion the benefits of the accreditation program outweigh the financial costs required for my organisation to participate. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 40% 30% 20% 10% ALL AGPAL ACSAA ACHS 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

The benefits of accreditation: Public disclosure of accreditation information is supported by many stakeholders but more work needs to be undertaken to know how and what information should be made available. References: Greenfield, D., Hinchcliff, R., Pawsey, M., Westbrook, J. and Braithwaite, J. (2013) The public disclosure of accreditation information: practices and perceptions of opportunities and challenges, Health Policy, 113(1-2): 151-159.

Percentage Question 54: In my opinion accreditation results should be made public. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

Percentage Question 56: In my opinion the public should be better informed about accreditation outcomes and what they mean. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 70% 60% 50% 40% 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA Response

40% Question 20: In my opinion the accreditation program is well regarded by healthcare consumers. (N_ALL = 489; N_AGPAL = 210; N_ACSAA = 179; N_ACHS = 100) 30% 20% ALL AGPAL ACSAA ACHS 10% 0% Strongly disagree Disagree Neutral Agree Strongly agree NA

The benefits of research collaborations investigating accreditation programs: Accreditation programs the world over have similar characteristics and face common challenges. References: Braithwaite, J., Shaw, C., Moldovan, M., Greenfield, D., Hinchcliff, R., Mumford, V., Kristensen, M., Westbrook, J., Nicklin, W., Fortune, T. and Whittaker, S. (2012) Comparison of health service accreditation in low- and middle-income countries with those in higher income countries: a cross sectional study, International Journal for Quality in Health Care, 24(6): 568-77.

Key challenges Role of government in accreditation schemes Financial viability of schemes Ongoing stakeholder acceptance and engagement Peer-to-peer or professional surveyors Reliability of surveyors and surveys

Now, are you an optimist or pessimist? Do you think this means we are improving healthcare and developing the evidence base? Or is it a lot of work for little value?

What else could we do to improve the accreditation field?

A/Prof David Greenfield, PhD FAAQHC FISQua Centre for Clinical Governance Research, Australian Institute of Health Innovation, UNSW Medicine, Sydney, Australia Email: d.greenfield@unsw.edu.au Mobile: +61419 462 847 Web: www.aihi.unsw.edu.au

From November 2014 I will be based at Macquarie University, Sydney, Australia http://www.mq.edu.au A/Prof David Greenfield, PhD FAAQHC FISQua Centre for Clinical Governance Research, Australian Institute of Health Innovation