Evaluation of the Continuing Competence Framework

Evaluation of the Continuing Competence Framework Report prepared for the Nursing Council of New Zealand October 2010 6303_nurse_competency copy copy.indd 1

2 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Nursing Council of New Zealand PO Box 9644, Wellington 6011 www.nursingcouncil.org.nz Published October 2010 Nursing Council of New Zealand 2010 ISBN 978 0 908662 34 0 6303_nurse_competency copy copy.indd 2

NURSING COUNCIL OF NEW ZEALAND 3 Table of Contents Foreword 7 Research team and authors 8 Acknowledgements 9 Executive Summary 10 Key research findings and recommendations 10 Chapter One Background 13 1.1 Introduction 13 1.2 The purpose of professional regulation 13 1.3 Legislation 14 1.4 Continuing competence 15 1.5 Chapter descriptions 16 1.6 Conclusion 16 Chapter Two Literature Review 17 2.1 Introduction 17 2.2 Competence and continued competence 17 2.3 Competence frameworks 18 2.4 Professional standards and competence assessment 19 2.5 Continuing competence indicators 21 2.6 Summary of findings 22 2.7 Conclusion 23 Chapter Three Research Design and Methods 24 3.1 Introduction 24 3.2 Research Design 24 3.3 Methods 25 3.4 Ethical approval 28 3.5 Limitations of the research 28 3.6 Conclusion 29 Chapter Four Phase One Results, Document Review and Analysis 30 4.1 Document Review 30 4.2 Framework for document analysis 30 4.3 History 30 4.4 Policy for recertification and policy for CCF 32 4.5 Council statistics 34 4.6 Summary of findings from the document review and policy analysis 36 4.7 Conclusion 36 6303_nurse_competency copy copy.indd 3

4 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Chapter Five Phase Two Results, Qualitative Interview Data 37 5.1 Introduction 37 5.2 Thematic category - Competence 38 5.3 Thematic category - The role of the Council 42 5.4 Thematic category - Recertification audit process 43 5.5 Summary of findings from the interviews 47 5.6 Conclusion 48 Chapter Six Phase 3 Results, Quantitative Questionnaire Data 49 6.1 Introduction 49 6.2 Demographic data 49 6.3 Competence and fitness to practise 56 6.4 Recertification audit 60 6.5 Professional Development and Recognition Programmes 65 6.6 Summary of findings from the questionnaire 67 6.7 Conclusion 68 Chapter Seven Phase Four, Discussion, Conclusion and Recommendations 69 List of tables 7.1 Introduction 69 7.2 Data triangulation and discussion 69 7.3 Research outcomes and discussion 72 7.4 Key research findings 76 7.5 Conclusion 78 References 79 Table 1. Continuing competence indicator requirements for nurses across six regulatory jurisdictions 20 Table 2. Council objectives addressed in the evaluation 28 Table 3. Analysis of key Council policy documents associated with the Continuing Competence Framework 33 Table 4. Participation rates and sample size 49 Table 5. Cross-tabulation highest qualification by scope of practice 50 Table 6. Cross-tabulation of current employment setting by scope of practice 53 Table 7. Cross-tabulation for current area of nursing practice by scope of practice 55 Table 8. Responsibility for maintaining continuing competence to practise 59 Table 9. Understanding of APC self-declaration questions 60 Table 10. Have you been a peer assessor? 61 Table 11. When you were audited did you receive written information about? 62 Table 12. Cross-tabulation receipt of audit information by audit year 63 Table 13. Understanding of how to provide evidence for recertification audit 64 Table 14. Following submission of audit documentation 64 Table 15. Satisfaction with recertification audit documentation, communication and process 65 Table 16. Cross-tabulation of employment setting by Should PDRPs be compulsory? 66 6303_nurse_competency copy copy.indd 4

NURSING COUNCIL OF NEW ZEALAND 5 List of figures Figure 1. Practising certificates issued 34 Figure 2. Recertification audit trends and competence notifications 35 Figure 3. Representation of questionnaire respondents 50 Figure 4. What is your highest qualification? overall group response 51 Figure 5. Current employment setting overall group response 52 Figure 6. Current nursing practice area overall respondent group 54 Figure 7. Indicators that provide the best evidence of competence to practise 56 Figure 8. Indicators that provide the best evidence of continuing professional development 57 Figure 9. The CCF provides a mechanism to ensure nurses are competent and fit to practise 58 Figure 10. Responsibility for maintaining continuing competence to practise 58 Figure 11. Peer assessor 61 Figure 12. Recertification audit distribution of respondents by audit year 62 Figure 13. Should Professional Development and Recognition Programmes be compulsory? 65 Figure 14. Levelled on a PDRP Have access to PDRP? 67 Table of statutes Health Practitioners Competence Assurance (HPCA) Act 2003 14 Nurses Act 1977 14 Nurses Registration Act 1901 14 Nurses Regulations, 1986 14 Table of cases Health Care Complaints Commission v Bruce Litchfield,(1997, 41 NSWLR 630) 70 Condon, NMT230206JHC in NSW NMB (in press due for publication 2010) Professional Conduct casebook 2nd edition NSW NMB: Sydney 70 6303_nurse_competency copy copy.indd 5

6 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 6303_nurse_competency copy copy.indd 6

NURSING COUNCIL OF NEW ZEALAND 7 Foreword The Nursing Council of New Zealand (the Council) established the Continuing Competence Framework in 2004 to meet the statutory requirements of the Health Practitioners Competence Assurance Act 2003 (the Act). The Continuing Competence Framework details the Council s requirements for ensuring the ongoing competence of nurses for the purposes of public safety. The Council began a major research project in July 2009 to determine if the way we currently measure continuing competence through practice hours, professional development hours and assessment of competence is the best way to determine continuing competence. These measures reflected the best thinking when the Health Practitioners Competence Assurance Act (2003) was introduced and five years on a review is appropriate. The Council has a commitment to evidence-based policy and practice, and to ensuring, as far as practicable, that its processes are underpinned by research and evaluation. To this end the Council determined to undertake an evaluation of the Continuing Competence Framework. A search identified that although there is research and practice literature related to the establishment, methodology and content of competence frameworks and programmes, there appears to be little research to date on evaluation of these methodologies. Consequently the Council believes this research will make a significant contribution to knowledge in the area of regulation policy. The Council is pleased to release this evaluation report on the framework. There was generally positive feedback about the framework and the current indicators of continuing competence the Council is using. There was also support for the recertification audit process as a mechanism for ensuring the framework is being adhered to by individual nurses. There are some recommendations for changes to the recertification process and some issues for further consideration for the Council related to the concept of continuing competence. Some respondents did not fully understand the respective roles of the Council, the employer and the individual nurses in ensuring competence to practise. As a result of the evaluation, the Council will not be changing the indicators that form the basis of the Continuing Competence Framework but will be implementing some improvements to the recertification audit documentation and processes. The Council thanks the research team for this evaluation and also the nurses who participated in the research by interview or survey. Margaret Southwick Chair of the Nursing Council of New Zealand 6303_nurse_competency copy copy.indd 7

8 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Evaluation of the Continuing Competence Framework This evaluation research has been commissioned by the Nursing Council of New Zealand. The research was conducted in an eight-month timeframe and was completed on 24 March 2010. The research team is listed below: Research team and authors Rachael Vernon (Lead Researcher) RN BN MPhil (Dist.)(Nursing) PhD Candidate (USYD) Head of School of Nursing Faculty of Health and Sport Science EIT Hawke s Bay Taradale, Hawke s Bay New Zealand Professor Mary Chiarella RN CM LLB(Hons) PhD (UNSW) Sydney Nursing School University of Sydney Sydney, New South Wales Australia Dr Elaine Papps RN BA MEd(Dist.) PhD Health, Education and Research Consultant Hastings, Hawke s Bay New Zealand Professor Denise Dignam RN BA PhD (Massey) Associate Dean (Teaching and Learning) Faculty of Nursing, Midwifery and Health University of Technology Sydney Sydney, New South Wales Australia 6303_nurse_competency copy copy.indd 8

NURSING COUNCIL OF NEW ZEALAND 9 Acknowledgements The research team would like to thank all who participated in this evaluation and contributed to the findings. Their valuable contributions have enabled the research team to comprehensively evaluate the Continuing Competence Framework. Disclaimer This publication is adapted from a research report commissioned by the Nursing Council of New Zealand. Please note that the views expressed in this document are those of the respondents and authors, and do not necessarily represent the Nursing Council s views. Throughout this evaluation the researchers have received data from a variety of sources. The researchers have assumed the accuracy of all data provided to them during the course of the evaluation. The researchers have not independently audited or otherwise checked the veracity of such data. Accordingly, the researchers undertake no responsibility in respect of the veracity of any data provided to them, upon which any part of the evaluation or this report is based. Citation Vernon, R., Chiarella, M., Papps, E. & Dignam, D. (2010, October). Evaluation of the continuing competence framework. Wellington: Nursing Council of New Zealand. 6303_nurse_competency copy copy.indd 9

10 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Executive Summary Background The Continuing Competence Framework (CCF) was introduced in 2004 following the enactment of the Health Practitioners Competence Assurance (HPCA) Act 2003. This evaluation research was commissioned by the Council to: 1. explore the validity of the stipulated hours of professional development and days/ hours of practice over a three-year period, as indicators of competence; 2. provide information on the efficacy of undertaking a random audit of five percent of the nursing workforce to meet recertification requirements; 3. document and track the different forms of written evidence that are currently acceptable to the Council to demonstrate competence; 4. identify issues related to peer assessment of competence; 5. develop a framework to enable the Council to complete a further evaluation in five years time. Research design and methods A sequential mixed-methods evaluation was designed to be completed in four phases with each sequential phase serving to inform the basis for the next phase of data collection and analysis. A literature review was undertaken and three sets of evaluation data were collected. These included: a historical review and analysis of the documentation during the development and implementation of the CCF to the present day; qualitative interviews with 26 key stakeholders and nurses ; and a quantitative survey of 1,157 nurses on the Council s active register. The results of these three sets of data were triangulated to address the required outcomes for the Council. Key findings The Continuing Competence Framework (CCF) The overwhelming consensus of key stakeholders was that the CCF is a critical and important mechanism to ensure nurses are fit and competent to practise. Seventy-six percent of survey respondents believe the Council s CCF and processes for renewing practising certificates provide the mechanism to ensure nurses are competent and fit to practise. There is historical evidence that the development of the CCF was well researched and included extensive consultation with stakeholders. Research outcomes 1. Explore the validity of the stipulated hours of professional development and days/hours of practice over a three-year period, as indicators of competence. following a number of consultative processes, the Council made a decision in August 2004 for three indicators of competence: 1. practice hours (minimum of 450 hours/60 days in past three years); 2. professional development (minimum of 60 hours in the past three years); 3. assessment of competence to practise against the Council s competencies for the relevant scope of practice. data from interviews and the survey indicate there is general satisfaction with the stipulated hours for professional development and practice. 6303_nurse_competency copy copy.indd 10

NURSING COUNCIL OF NEW ZEALAND 11 nurses were asked to rank the indicators which they believed provided the best evidence of continuing competence to practise. The indicator of competence to practise ranked best by 52% of respondents (n = 470) was the combination of the self-declaration, evidence of practice hours and evidence of ongoing professional development. 2. Provide information on the efficacy of undertaking a random audit of five percent of the nursing workforce to meet recertification requirements. Five percent of nurses renewing their practising certificates annually are randomly selected for individual recertification audit. This figure is pragmatic, as no single piece of evidence exists to prove it is a valid representation of the New Zealand nursing workforce. However, statistical findings from the recertification audits conducted over the past five years suggest that the five percent measure is appropriate and effective. Recertification audit is generally considered by respondents to be an important quality indicator that provides a measure of validity and reliability to the CCF. 3. Document and track the different forms of written evidence that are currently acceptable to the Council to demonstrate competence. The current documentation provided to nurses selected for recertification audit was examined. The interviews identified a number of quality improvement issues with regard to access to recertification audit documentation, guidelines and submission and assessment of audit materials. Issues were identified with regard to the validity and reliability of the self-declaration, selfassessment, peer assessment and assessment of Council evidence for the recertification audits. Recommendations That the CCF documentation available to nurses is revised to provide more explicit and detailed guidelines with regard to CCF evidential requirements, assessment processes, recertification audit process including timeframes for the recertification assessment and notification of the outcomes of the audit. Explore the potential to provide recertification audit material and guidelines in both hard copy and electronic formats, with the provision for participants to enter data directly onto electronic forms. Investigate a system of electronic submission recertification audit data. Develop a system of electronic tracking of recertification audit documents, accessible to nurses who are participants in the audit process. Instigate a clear internal moderation process to improve inter-rater reliability and transparency of audit processes. 4. Identify issues related to peer assessment of competence The nurses selected for the recertification audit are required to complete two of the following forms of assessment: self, peer or senior nurse. There is a lack of clarity and some confusion in regard to the required standard for assessment of competence. In response to a question about whether survey respondents had ever been asked to be a peer assessor, 21% of the overall respondent group indicated they had been a peer assessor. Of the peer assessor group, 25% indicated they were not provided with information about process, 18% were not provided with documentation about the scope of practice, 10% were not provided with assessment forms, 6% indicated their assessment was not based on evidence, and 14% did not discuss the assessment with the colleague they assessed. 6303_nurse_competency copy copy.indd 11

12 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Recommendations Provide on all CCF-related documentation (including the Council website) a clear and consistent definition of what constitutes continuing competence with explicit criteria of how continuing competence may be assessed. Revise the criteria and guidelines for the selection of peer assessors. Provide peer assessors with guidelines for the peer assessment process in addition to clear and explicit assessment criteria. Provide documentation options in hard copy or electronic formats. Other findings It was reported by some interviewees that there was a lack of clarity with regard to the role, responsibility and accountability of the Council, the individual nurse and the employer in terms of ensuring continuing competence to practise and public safety. It was reported that some interviewees indicated that there was confusion between the evidential requirements of the CCF recertification audit, PDRP, the evidential requirements of the PDRP and the role of the Council, employers and the individual nurse in this regard. Questions were raised with regard to the verification and legal status of the self-declaration on the Application for Practising Certificate form. Recommendations Revise the CCF and Recertification Audit policy documentation to include a clear purpose statement and policy framework principles. Improve and make overt the public nursing profile of the Council with regard to its role and responsibility as the regulatory authority for nurses in New Zealand. Differentiate and communicate the Council s expectations with regard to the responsibility of individual nurses, employers and the profession in terms of the requirements of the CCF and the HPCA Act 2003. Clearly articulate and communicate the Council s role and responsibilities with regard to PDRP. Provide a clear and more comprehensive definition of the status of the self-declaration on the Application for Practising Certificate form. Reformat the Application for Practising Certificate form to make the crucial information with regard to the self-declaration more explicit and obvious. Clearly articulate the penalties for providing false and misleading information on all documentation related to the CCF. A detailed summary of research findings is provided at the conclusion of each research phase in chapters two, four, five, six and seven. 6303_nurse_competency copy copy.indd 12

NURSING COUNCIL OF NEW ZEALAND 13 Chapter One Background 1.1 Introduction This document sets out the findings from research undertaken on behalf of the Council to evaluate its Continuing Competence Framework (CCF), which was implemented in 2004 following enactment of the Health Practitioners Competence Assurance (HPCA) Act 2003. The report begins with an overview of the legislative history governing the regulation of the nursing profession in New Zealand and moves on to provide an overview of a number of literature reviews relating to continuing competence and regulation. The report then sets out the method and research findings, finally triangulating the data to provide a number of research outcomes and recommendations for the ongoing development and future evaluation of the CCF by the Council. The data do identify some areas of confusion in relation to the CCF. It is not the intention of the report to dwell on matters that are outside the jurisdiction of the governing legislation, such as the Professional Development and Recognition Programme (PDRP). However, the report touches briefly on the confusion that the interview data seem to indicate exists between the evidential requirements of the PDRP and the evidential requirements of the CCF recertification audit. Nurses who participate in a Council-approved PDRP are exempt from the recertification audit process. The report seeks to clarify the situation with regard to the Council requirements in relation to the required standard of competence as defined in s5(1) HPCA Act 2003. The report is also concerned to explain that the lens through which the work of any regulatory authority is viewed is that of protective jurisdiction and this chapter will commence with an explanation of the purpose of professional legislation and the nature of a protective jurisdiction. 1.2 The purpose of professional regulation It is often misunderstood by nurses and indeed other health professionals in relation to their registering authority that the role of the legislation is protective. The legislation and therefore the institutions, roles and committees created by it exist to protect the public from the risk of harm, rather than to protect the interests of the professions so regulated. The functions and powers of a registering authority are defined in legislation and establish a form of regulatory regime known as a protective jurisdiction (Staunton & Chiarella, 2008, pp. 213-214). This form of occupational regulation provides: a barrier to entry to the professions by untrained persons; a mechanism for standards of education and practice to be established and enforced; and an avenue for consumers to have complaints against practitioners addressed (National Nursing and Nursing Education Task Force, 2006). Of course this need for professional regulation is always a balancing act between public safety and the risk of exclusionary practices or elitism. In New Zealand the Director-General of Health has recently completed a review of the operation of the HPCA Act 2003. It is reported that concerns were raised during the review that New Zealand has a proliferation of registration authorities (New Zealand Ministry of Health, 2009). As a result the Director-General has recommended that the Ministry of Health should review the criteria it uses to advise the Minister whether regulation is justified. A discussion document has been put out whose purpose is to: outline the policy principles that are relevant to regulating health professions; discuss the Ministry s current criteria for regulation and those used in similar jurisdictions; propose revised criteria to assist the Ministry in advising the Minister whether a profession poses a risk of harm or it is otherwise in the public interest to regulate that profession (New Zealand Ministry of Health, 2010). 6303_nurse_competency copy copy.indd 13

14 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Whilst this is outside the scope of this review it is important to recognise that the public interest is always the primary issue of importance in such reviews. The role of the Council, the regulatory authority established to administer the legislation in relation to nurses in New Zealand, is, among other things, to establish and maintain standards of practice. Any health practitioner who is concerned about another health practitioner s practice and who considers the standard of practice may pose a risk of harm to the public may notify the Registrar of the relevant authority (s34 HPCA Act 2003). There are also provisions for both the public and specified health providers to notify the Registrar about health concerns that may affect a health practitioner s ability to practise (s45 HPCA Act 2003). 1.3 Legislation Nurses have been regulated in New Zealand for over 100 years. However, the enactment of the Health Practitioners Competence Assurance (HPCA) Act 2003 in New Zealand heralded a significant change for all health practitioners, including nurses. This new legislation brought 15 health professional groups under one act and repealed 11 separate statutes relating to 13 individual regulatory authorities. It is clear to see that the HPCA Act sits within a protective jurisdiction, as its principal purpose is stated as being: to protect the health and safety of members of the public by providing for mechanisms to ensure that health practitioners are competent and fit to practise their professions (HPCA Act, 2003, s1). An extensive review and collation of the literature indicated there was a paucity of research relating to the implementation of the HPCA Act 2003 and the subsequent Nursing Council of New Zealand (the Council) requirements to ensure nursing professionals are competent and fit to practise. The legislation previously regulating the registration of nurses in New Zealand, the Nurses Act 1977 and its subsequent amendments, was the last iteration of separate professional legislation regulating nurses and had its origins in the Nurses Registration Act 1901. Between 1901 and 1977 this legislation, although addressing public safety, was silent on the issue of competence (Burgess, 2008; Papps, 2002). Instead, it referred to other terms such as fitness and properness. These terms are arguably more associated with the notion of suitability to practise rather than capability or ability to practise, which is more the focus of competence. Furthermore the terms fitness and properness were not clearly defined under the Nurses Act 1977, as they were considered matters for judgement by the Council (Nurses Regulations, 1986). Competence is still not defined per se in the HPCA Act. However, the Act is specific in its requirements of what constitutes competence, fitness to practise and quality assurance, and clearly specifies the legislative functions of the 15 regulatory authorities, including the Council. In addition, the Council has defined competence as the combination of skills, knowledge, attitudes, values and abilities that underpin effective performance as a nurse (NCNZ, 2009). Additionally the Act requires all authorities to specify scopes of practice (HPCA Act 2003, s11) including the requisite qualifications for each scope (HPCA Act 2003, s12). In 2004 the Council specified and gazetted four scopes of practice, each with registration and competence requirements - Nurse Assistant, Enrolled Nurse, Registered Nurse and Nurse Practitioner. Within each legislated scope of practice, domains of practice and underpinning competencies are specified which are required to be translated into programme curricula. The individual registration authorities have the legal responsibility for monitoring the educational institutions accredited and approved to provide programmes leading to registration in a particular scope of practice (HPCA Act 2003, s12(4)). All Nurse Assistant 1, Bachelor of Nursing and Master of Nursing programmes (for preparation of Nurse Practitioners), are under the scrutiny of the Council which approves, monitors, and audits all nursing programmes for preparation for entry to the register of nurses including competency based return to practice programmes. Currently the Council also approves and monitors some nursing programmes at graduate and postgraduate level, which do not lead to registration in a scope of practice - for example, the nurse entry to 1The research was undertaken in 2009 when nurse assistant was one of the scopes of practice for nurses. This scope of practice was replaced on 31 May 2010 and nurse assistants became enrolled nurses. 6303_nurse_competency copy copy.indd 14

NURSING COUNCIL OF NEW ZEALAND 15 practice (NETP) programmes and some postgraduate certificate and postgraduate diploma programmes. The HPCA Act affords regulatory authorities significant power in relation to professional competence and requires them to set standards of clinical competence, cultural competence and ethical conduct to be observed by health practitioners of the profession (HPCA Act 2003, s118(i)). These aspects are considered cornerstones of health professional education and indeed the overarching notion of public protection, and are afforded equal significance under the HPCA Act (HPCA Act 2003, p1 s4(6)). 1.4 Continuing competence In 2004 following the enactment of the HPCA Act 2003 the Council established and implemented a continuing competence framework (CCF) (NCNZ, 2004a), the primary purpose being to provide mechanisms to ensure nurses are competent and fit to practise their profession as stipulated in s1 HPCA Act 2003. This new requirement introduced a significant change to the process for ongoing nurse registration. Before the introduction of this statute the only requirement for a nurse to renew their registration was to pay their annual fee and sign a renewal form. There was no requirement either to declare competence or to provide evidence of being competent (Papps, 1997). The process for the ongoing monitoring of the continuing competence of nurses, once registered and in practice, is now the responsibility of the Council. The Council now has the authority to decline to issue an annual practising certificate (APC) if the applicant has, at any time, failed to meet the required standard of competence, failed to comply with conditions, not completed an ordered competence programme, not held an APC (or practised) for three years preceding application (HPCA Act 2003, s27(1)(a)). The required standard of competence is defined under the HPCA Act 2003 s5(1) as the standard of competence reasonably to be expected of a health practitioner practising within that health practitioner s scope of practice. The Council initially signalled the introduction of competence-based practising certificates in 1994 with the publication of the Strategic Plan 1st April 1994 to 31st March 1997 (NCNZ, 1994). This and further review of the historical progress of competence-based practice will be discussed in chapter four. These original ideas of the Council varied from the guidelines implemented by the Council in 2004, which specify three indicators of continuing competence associated with the maintenance of a personal professional profile and self-declaration of competence, which are required on an annual basis. The current requirements for the CCF are: A Evidence of ongoing professional practice Nursing practice is using nursing knowledge in a direct relationship with clients or working in nursing management, nursing administration, nursing education, nursing research, nursing professional advice or nursing policy development roles, which impact on public safety (minimum of 60 days or 450 hours within the last three years). B Evidence of ongoing professional development Ongoing education (minimum of 60 hours in the last three years, relevant to work environment and practice as a nurse). C Evidence of meeting the Council s competencies for the nurse scope of practice Self-declaration that states the individual meets the Council s competencies for their scope of practice, i.e. Registered Nurse, applied to the area or context in which you practise (NCNZ, 2006). 6303_nurse_competency copy copy.indd 15

16 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 1.5 Chapter descriptions Chapter two reviews and summarises national and international literature relating to the development and implementation of CCFs. It focuses on the conceptualisation of competence, indicators and validity as these relate to public safety. The literature will also be used to inform the central concerns of the key research outcomes in chapter seven. Chapter three sets out the methods used to undertake this evaluation research and includes a justification for the research design and the ethical considerations taken. The major components of the research process include three phases of inquiry: a document review and analysis; a series of interviews with a purposive sample of nursing opinion leaders who had direct knowledge and experience of the CCF; and a purpose-designed questionnaire sent out to a convenience sample of nurses who had previously agreed to participate in surveys and had provided their email addresses to the Council. The final phase of the research involved a triangulation of these data sets. Chapters four, five and six present the results of the three phases of the research and situate these results within the research questions that formed the research design. Chapter seven triangulates the data to address the required research outcomes identified by the Council and presents a summary of the key research findings and recommendations. 1.6 Conclusion This chapter has provided an introduction and background to the evaluation of the CCF. As outlined the methodology and findings will be presented in the chapters that follow. 6303_nurse_competency copy copy.indd 16

NURSING COUNCIL OF NEW ZEALAND 17 Chapter Two Literature Review 2.1 Introduction It is commonly agreed that nurses hold a position of trust in society and have a responsibility to be competent, with opinion polls repeatedly demonstrating this trust (Jones, 2005; McManus & Sieler, 1998). Current registration requirements in the jurisdictions of most developed countries contain an expectation that nurses will not only be competent to practise nursing on registration, but will maintain that competence in respect of their chosen field or scope of practice as they develop in their careers and renew their registration (Chiarella, Thoms, Lau & McInnes, 2008). The implementation of the Health Practitioners Competence Assurance Act (HPCA Act) 2003 brought with it a shift in emphasis, a greater focus on health practitioners competence and increased consumer awareness. Before this, the system for regulating health professionals in New Zealand once they had qualified in their respective discipline and demonstrated they were fit to practise was registration for life. For nurses, practising certificates were renewed by application to the Council and by paying an annual fee (Vernon, 2010; Vernon, Chiarella & Papps, 2010). The HPCA Act 2003 requires the regulatory authority, the Nursing Council of New Zealand (the Council), to set the requirements for programmes that lead to entry to the register of nurses and also the requirements for, and monitoring of, the continuing competence of nurses once registered. The Council is the product of a protective jurisdiction and its Continuing Competence Framework (CCF) is a mechanism of recertification that allows nurses to demonstrate annually that they remain competent and fit to practise. These changes have been the subject of considerable debate in the NZ context over the years since the introduction of the CCF, but the prevailing view of government is that trust alone is insufficient to guarantee individuals are competent and fit to practise in their profession (HPCA Act 2003, s1). For the large majority of nurses and also in the eyes of the New Zealand public, this mechanism of recertification provides reassurance and reinforcement of their continued competence and performance. However, for a minority it also provides a mechanism for identifying competence issues and the opportunity to address them. This chapter reviews and summarises national and international literature relating to the development and implementation of CCFs. Issues with regard to competence, competence assessment and validity of competence indicators will be examined. Key findings are summarised and presented at the conclusion of the chapter. An extensive search (1990 2010) of literature relating to nursing, medical and allied health disciplines nationally and internationally was completed using the following search engines and data bases - Google, ProQuest, CINAHL and Ovid. Key search terms were - competence, nurse competence, continued[ing] competence, competence framework, competence indicators, continued[ing] professional development, continued[ing] nursing education, regency of practice. Of particular note were seven international reviews (ANMC, 2007; Bryant, 2005; Chiarella, et al., 2008; CNA, 2000; EdCaN, 2008; FitzGerald, Walsh & McCutcheon, 2001; NCSBN, 2009) related to continuing competence in nursing, completed since 2000. In addition the regulatory authorities in the following countries (Australia, United States of America, United Kingdom and Canada) were contacted directly by email and/or telephone to ascertain the continuing competence models currently being used in each jurisdiction. 2.2 Competence and continued competence There is still considerable debate about the conceptualisation of competence and its assessment (Bryant, 2005; Chiarella, 2006; Chiarella, et al., 2008; Cowan, Norman & Coopamah, 2005; EdCaN, 2008; Hendry, Lauder & Roxburgh, 2007; Pearson, FitzGerald, Walsh & Borbasi, 2002). A common theme in the debate about the definition of competence is the need for flexibility, given the multifaceted nature of nursing practice and the diversity of practice settings. Pearson et al. (2002) contend that the insight of individuals in relation to their expertise and limitations is critical, and argue there is a direct correlation between lack of insight and potential or actual unsafe practice. 6303_nurse_competency copy copy.indd 17

18 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK The Council has defined competence as: the combination of skills, knowledge, attitudes, values and abilities that underpin effective performance as a nurse (NCNZ, 2009). To determine whether a nurse has maintained the required standard of continuing competence and whether a nurse requires a practising certificate the Council has also defined nursing practice as: using nursing knowledge in a direct relationship with clients or working in nursing management, nursing administration, nursing education, nursing research, nursing professional advice or nursing policy development roles, which impact on public safety (NCNZ, 2010). The definition stated by the Council reflects the principles articulated by the International Council of Nurses (ICN) (Bryant, 2005) and is consistent with other international definitions, including the work of the Australian Nursing and Midwifery Council (ANMC), until 2010 the peak regulatory body in Australia, the National Council of the State Boards of Nursing (NCSBN), United States of America, and the Canadian Nurses Association (CNA). The ANMC developed a number of advisory papers on continuing competence, defining competence as: the combination of skills, knowledge, attitudes, values and abilities that underpin effective and/or superior performance in a profession/occupational area and context of practice and continuing competence as: the ability of nurses and midwives to demonstrate that they have maintained their competence to practise in relation to their context of practice, and the relevant ANMC competency standards under which they gain and retain their licence to practise (ANMC, 2009). The National Council of the State Boards of Nursing (NCSBN) (US) has also produced a number of valuable discussion documents and has previously defined competence as: the application of knowledge and the interpersonal decision-making required for the practice role, within the context of public health (NCSBN, 2009). Similarly the Canadian Nurses Association defines competence as: the ability of a registered nurse to integrate and apply the knowledge, skills, judgement and personal attributes required to practise safely and ethically in a designated role and setting (CNA, 2000). It is also evident from the literature that confusion exists between the meaning of competence, performance and continued competence (Flanagan, Baldwin & Clarke, 2000; McMullan, 2006). Competency standards have been developed in New Zealand and internationally as a way of differentiating and standardising the variations in scopes and levels of practice within the nursing profession (Chiarella, et al., 2008; Vernon, 2010). In New Zealand the four legislated scopes of practice - Nurse Assistant, Enrolled Nurse, Registered Nurse and Nurse Practitioner - are clearly defined, each with their own set of competencies and standards. These competencies also form the basis for development of curricula and assessment tools, and include cultural competence. Whilst standards for entry-level competence are clearly articulated and relatively well understood, the interpretation by nurses of what constitutes continued competence is not. 2.3 Competence frameworks The principal function of a CCF is to act as a quality assurance mechanism to ensure health professionals are competent in their practice and thereby protect the public (ANMC, 2007; CNA, 2000; Goodridge, 2007). For health consumers and employers, CCFs and their associated standards and monitoring activities offer a level of assurance that practitioners are competent whilst providing a mechanism for identifying those who are not. 6303_nurse_competency copy copy.indd 18

NURSING COUNCIL OF NEW ZEALAND 19 As identified in the literature (ANMC, 2007; Bryant, 2005; CNA, 2000) frameworks are tools that have a role in regulating and guiding the profession by setting the standards for competence assessment and ensuring consistency in the monitoring and ongoing assessment of competence (Pearson, FitzGerald, Walsh, et al., 2002). They have a clear purpose in terms of public protection. However, literature suggests that if their purpose is also to promote lifelong learning then this must be clearly articulated (ANMC, 2007; Campbell & MacKay, 2001; Goodridge, 2007) as it will influence the level of assessment required. Issues with regard to standardising nursing practice (McGrath, et al., 2006) highlight the distinctive differences between entry level/beginning competence and continued competence (Benner, 1984; Chiarella, et al., 2008; FitzGerald, et al., 2001; Hendry, et al., 2007; Pearson, FitzGerald, Walsh, et al., 2002). Dolan (2003) stresses the importance of taking account of the real-world setting and reiterates the aim of producing a competent nurse is to ensure a high quality of patient care. The fact that fitness to practise goes beyond mere adequacy of knowledge and skills, and should take account of the complexities of nursing practice (Gibson & Soanes, 2000; National Nursing Research Unit, 2009; Pearson, FitzGerald, Walsh, et al., 2002), is highlighted when reviewing the various definitions of competence. There is general agreement in the literature that frameworks, standards for, and assessment of, continuing competence should relate to the individual s particular scope of practice and area of practice (ANMC, 2007). The ANMC is currently formulating a national framework for continuing competence, similar to the Council s CCF, to provide nurses, midwives and their employers with a means to demonstrate ongoing competence to practise (ANMC, 2007). The ANMC draft framework (2007) references the Council s CCF and incorporates the same three indicators of continued competence: self-assessment against the relevant competencies; practice hours; and continuing professional development (CPD). The new National Nursing and Midwifery Board of Australia has circulated the future continuing competence framework for consultation in preparation for national registration of nurses in Australia, as of 1st July 2010. A number of regulatory authorities have developed frameworks similar to that of New Zealand and Australia, and all report similar experiences and challenges in terms of monitoring and validating competence indicators. Table 1 reflects a summary of continuing competence framework requirements for nurses across five countries 2.4 Professional Standards and competence assessment Competence assessment of practising nurses is identified as crucial to maintaining professional standards (McMullan, 2006) and as such has a role in professional regulation (Chiarella, et al., 2008). As identified by EdCaN (2008) the literature identifies a tension between academic qualifications and a professional s competence to practise (Gibson & Soanes, 2000). This tension is complicated by the lack of consensus about the meaning of competence, and how and what aspects of competence should be measured. It is suggested that any lack of consensus increases the potential for confusion and repetition as nurses attempt to meet the requirements of a number of different systems (Storey & Haigh, 2002). There is general agreement in the literature that competence assessment in nursing cannot solely be based on demonstration of theoretical knowledge or technical skills, but should also involve some inference about a candidate s attitudes and professional practice (EdCaN, 2008). Continued professional development (CPD) as an indicator of competence ensures the continued competence of an individual is relevant to current nursing practice (Meretoja, Isoaho & Leino-Kilpi, 2004). However, the most common indicator of competence in nursing practice is performance. There is considerable debate about the assessment and adequacy of performance as a valid indicator (FitzGerald, et al., 2001; McMullan, 2006). Whether demonstration of a particular skill or activity, in one area or on a particular day, is indicative of competence in all situations on any given day (Gibson & Soanes, 2000), and whether competence is directly observable in terms of performance of an activity (McGrath, et al., 2006; National Nursing Research Unit, 2009) are issues that are still the subject of debate. The literature suggests that observed competent performance of tasks can only be inferred, as the measurement of underpinning competencies requires evaluation of aspects such as behaviours, attitudes and insights that are not readily amenable to quantification (National Nursing Research Unit, 2009). Similar issues have been identified in relation to assessment of the different levels of nursing practice (Benner, 1984; Calman, Watson, Norman, Redfern, & Murrells, 2002). 6303_nurse_competency copy copy.indd 19

20 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Table 1 Continuing competence requirements for nurses across six regulatory jurisdictions in five countries New Zealand Nursing Council of New Zealand (National Framework) Australia National Nursing and Midwifery Board of Australia (National Framework proposed from 2010) United Kingdom Nursing and Midwifery Council (UKNMC) (National Framework) United States of America National College of State Boards of Nursing (NCSBN) (National Principles, Framework requirements vary across States and Territories) Canada Canadian Nurses Association (CAN) College of Registered Nurses of British Columbia College of Registered Nurses of Manitoba (National Principles, Framework requirements vary across States and Territories) Recertification of practising certificate annually Maintain a professional portfolio Self-declaration (self assessment against relevant competencies for practice) o Practice - minimum of 450 hours (60 days) in previous 3 years o Professional Development minimum of 60 hours in previous 3 years 5% Audited Annually Revalidation of registration annually Maintain a professional portfolio Formal self-declaration of competence annually Practice must have practised in previous 5 years or completed return to practice programme statutory declaration from individual or employer indicating hours spent in practice Continuing Professional Development (CPD) minimum of 20 hours annually 2% Audited Annually Required to renew registration every 3 years. Currently moving toward a system of annual revalidation Maintain professional portfolio Self-declaration complied with all Prep standards and signed notification of practice or intent to practice o Prep practice standard - minimum of 450 hours in previous 3 years or undertaken approved return to practice programme o Prep continuing professional development (CPD) standard - in previous 3 years No Audit % stated Revalidation of registration Self declaration, including self assessment of competence, declaration of criminal convictions, physical mental drug related issues that affect the ability to provide safe effective nursing care. Continuing Education credits Practice hours *Requirements vary between the legislative jurisdictions - e.g. New Hampshire Board of Nursing, 30 hours continuing education in previous 2 years and 400 hours practice in previous 4 years. No Audit % stated Revalidation of registration Self-declaration including self assessment Continuing education annual requirements o Report of CE activities and evaluation of learning needs o Development of a learning plan, report on previous plan o Peer feedback / review meetings Practice minimum of 1,125 hours in previous 5 years *Requirements vary between the legislative jurisdictions - General principles of the CAN and e.g. of two Canadian regulatory authorities No Audit % stated 6303_nurse_competency copy copy.indd 20

NURSING COUNCIL OF NEW ZEALAND 21 The literature indicates that in any competence assessment process the challenge is to ensure objectivity (Gibson & Soanes, 2000). Pearson, et al. (2002) note that measurement of competence is a form of regulation that may be limiting and failure to achieve competence in post-registration nursing can have a negative effect on the nurse, the assessor and the profession (Flanagan, et al., 2000). Standardisation of nursing practice through the development of generic competencies that do not take account of the specific context, or the diversity of practice environments, is cautioned against (McGrath, et al., 2006). Many assessments employed in continuing competence frameworks are based on self-assessment or direct observations by a peer, a mentor, a manager or an assessor, and include some level of subjectivity (FitzGerald, et al., 2001). Therefore the need for assessment approaches to encourage inter-rater reliability is critical (McGrath, et al., 2006). Numerous competence assessment tools are identified in the literature (Centre for Innovation in Professional Health Education and Research, 2007; EdCaN, 2008; FitzGerald, et al., 2001; Hendry, et al., 2007; McMullan, et al., 2003; Watson, Stimpson, Topping, & Porock, 2002). However, in general there is agreement that assessment of competence should include more than one competence indicator and assessment method (ANMC, 2007; CNA, 2000; EdCaN, 2008; McGrath, et al., 2006; Pearson, FitzGerald, Walsh, et al., 2002; Scott Tilley, 2008). Few articles describe competence assessment tools which provide approaches to ensure validity and reliability (McGrath, et al., 2006; Scott Tilley, 2008), although there is consensus that standardised assessment tools can be used to measure technical skills. However, decision-making and behavioural skills require a level of judgement from the assessor, as they are by nature subjective and difficult to quantify (Davis, Turner, Hicks, & Tipson, 2008; McGrath, et al., 2006). Pearson, et al. (2002) caution that the more subjective the nature of assessable item, the more difficult it is to specify a generic criterion for measurement. 2.5 Continuing competence indicators Whilst there is considerable discussion in the literature with regard to the conceptualisation and assessment of entry-level competence, there are few studies which address the issue of assessment and validity of competence indicators. The literature suggests that indicators of continuing competence are not easily defined and go beyond measurement of entry-level skills, and that a valid inference about continuing competence is not possible using a single indicator in isolation. Several authors (ANMC, 2007; Campbell & MacKay, 2001; CNA, 2000; EdCaN, 2008; FitzGerald, et al., 2001; Goodridge, 2007; Pearson & FitzGerald, 2001) have attempted to summarise the most commonly used indicators of continuing competence: 2.5.1 Self-assessment and self-declaration of competence generally this is a process of self-reflection/ assessment by an individual of their practice, set against the relevant regulatory standards/competencies for practice. Most commonly it involves the individual signing a self-declaration of competence. It is evident from the literature that whilst the individual is responsible for their own competence, there is also debate and a level of confusion about the employer s responsibility in terms of identifying, facilitating and supporting continued competence (ANMC, 2007; Campbell & MacKay, 2001; CNA, 2000; Goodridge, 2007). The main criticism of self-assessment is that it is subjective in nature and is reliant on the individual s insight and ability to assess critically. As such the assessment may lack validity unless linked with a formal feedback mechanism which promotes a connection between identified and actual practice weaknesses and learning needs relevant to the context of practice. 2.5.2 Recency of practice/hours of practice infers currency of knowledge and skills, and is quantifiable in terms of assessment of a skill or task and verification of hours of practice. However, used independently it is not an adequate indicator of continued competence or safety to practise (ANMC, 2007; Campbell & MacKay, 2001; FitzGerald, et al., 2001). 2.5.3 Continuing professional development (CPD) primarily concerned with the maintenance and updating of professional knowledge and is an indicator which appears in many CCFs (ANMC, 2009; CNA, 2000; EdCaN, 2008; FitzGerald, et al., 2001; NCNZ, 2004a; Nursing and Midwifery Council, 2008). CPD is considered to be a valid indicator and has the potential to improve currency of knowledge, skills, reflective activity and insight, but used independently it is not a reliable indicator of competence. 6303_nurse_competency copy copy.indd 21

22 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 2.5.4 Contribution to the profession participation in research, committees, policy development, quality assurance programmes and publication infers involvement in current practice and professional networks but does not infer competence or safety to practise (EdCaN, 2008; FitzGerald, et al., 2001). 2.5.5 Portfolio a tool used to record practice and develop an individual s reflective thinking/practice. It is subjective in nature and lacks inter-rater reliability. When used on its own it is not a reliable measure of competence or safety to practise (ANMC, 2007; EdCaN, 2008). 2.5.6 Peer review is identified as a feasible method of competence assessment, although it is time consuming for the individual and the reviewer, and can have issues of inter-rater reliability. It is cautioned that the peer reviewer must have a clear understanding of the criteria for assessment and the context of practice. There is ongoing debate in the literature as to the meaning of Peer. Should a peer reviewer have the same professional education, qualifications and scope of practice, and be in a similar position, or be a colleague with equal or higher status from another work area/discipline? (ANMC, 2007; CNA, 2000; Goodridge, 2007; Gopee, 2001) Regardless of the selected option, the peer review process must produce an auditable trail that demonstrates a valid assessment of competence that would meet requirements of public accountability (ANMC, 2007; CNA, 2000). 2.5.7 Performance appraisal evaluation of an employee by an employer/manager is generally undertaken to identify the ongoing competence of employees, to identify learning needs, promotion, salary increments, etc. Validity and reliability of performance appraisal is dependent upon the assessment mechanism, tools and criteria used. Used in conjunction with a formal Peer review of the individual s performance, it may be used to demonstrate continuing competence in practice (ANMC, 2007; CNA, 2000). 2.5.8 Objective Structured Competence Assessment or Evaluation (OSCE) there is debate in the literature as to the validity and reliability of simulated clinical skill assessments such as OSCEs in assessing continued competence (CNA, 2000; FitzGerald, et al., 2001; Goodridge, 2007). In addition the expense of administering OSCEs is high. 2.5.9 Examination this is a commonly used indicator in the United States. Fitzgerald, et al. (2001) report there is no research to support that it is an effective indicator of continued competence to practise. However, it does provide a standard form of assessment of knowledge for all registrants. 2.6 Summary of findings 2.6.1 Competence and continued[ing] competence Despite clear and reasonably consistent definitions of competence articulated by a number of nursing regulatory authorities a level of confusion about the conceptualisation of competence and the distinction between core and higher levels of competence, behaviours and insight is still apparent. 2.6.2 Continuing Competence Frameworks Promote consistency of continuing competence standards and assessment Provide a mechanism for the assessment of competence as a potential measure of public safety Demonstrate to the public that the regulatory authority and nursing profession are cognisant of and have mechanisms to assess the continued competence of the profession and ensure public safety Should be flexible, have relevance and be transferable to the differing levels of practice and settings in which nurses practise Should be mandatory for all members of the profession 6303_nurse_competency copy copy.indd 22

NURSING COUNCIL OF NEW ZEALAND 23 2.6.3 Communication Clear articulation of the purpose of the CCF is required (public protection or public protection and lifelong learning) Clear articulation of continued competence standards is required Staged introduction of CCF components is of benefit Involvement of all levels of the profession is essential for success Documentation related to the CCF is accessible and processes are transparent Web-based options should be available 2.6.4 Indicators of competence Self-assessment and self-declaration of competence, peer assessment, recency of practice and continuing professional development/education are the most commonly used indicators of continuing competence Recency of practice and CPD are quantifiable indicators. However, used independently they do not infer continuing competence to practise A combination of indicators is recommended no one indicator used independently can infer competence or safety to practise Valid measurement of indicators which are subjective in nature is difficult. Inter-rater reliability is a critical component of the assessment process 2.6.5 Assessment methods A variety of assessment methods and tools should be available Assessment tools should be user-friendly and relate to the individual practice context Tools and assessment criteria should relate directly to the standards for continued competence Clear communication/guidelines for assessment methods and tools should be available 2.6.6 Research Ongoing evaluation of the impact of CCFs is essential this should include outcomes for consumers, employers and nurses Validity of assessment methods included in CCF should be ongoing 2.6.7 Who is responsible? Confusion with regard to who is responsible for ensuring, demonstrating and facilitating continued competence the individual, the employing organisation, the professional body or the regulatory authority? CCFs need to identify clearly that their purpose is to ensure competence, the responsibility of individual nurses is to demonstrate competence and the employer s responsibility is to monitor competence 2.7 Conclusion The summary of findings from the literature mirrors those of previous reviews of the literature and should be considered in the context of the ongoing debate internationally, about the problems of ensuring continued competence. There has been little work done on obtaining consensus as to what might constitute best practice in demonstrating and assessing continued competence. Difficulties with regard to assuring valid and reliable assessment of continued competence in nursing have been highlighted. While a range of competence indicators and assessment tools were identified, few articles described rigorous approaches to ensure validity and reliability of competence assessment tools. The majority of studies are descriptive in nature, reporting predominantly qualitative findings. Limitations included small sample sizes and voluntary participation. This research seeks to evaluate the processes and tease out the understanding of and attitudes towards the Council s CCF in relation to its success in assuring competence under the HPCA Act 2003. Given the current lack of empirical evidence there is an opportunity for this research to contribute valuable information to the international literature. Chapter three presents the research design and methods used to evaluate the Council s CCF. 6303_nurse_competency copy copy.indd 23

24 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Chapter Three Research Design and Methods 3.1 Introduction The purpose of this research was to undertake an evaluation of the Council s Continuing Competence Framework to: 1. explore the validity of the stipulated hours of professional development and days/hours of practice over a three year-period, as indicators of competence. 2. provide information on the efficacy of undertaking a random audit of five per cent of the nursing workforce to meet recertification requirements. 3. document and track the different forms of written evidence that are currently acceptable to the Council to demonstrate competence. 4. identify issues related to peer assessment of competence. 5. develop a framework to enable the Council to complete a further evaluation in five year s time A comprehensive overview of the contemporary literature suggests that reliability and validation of competence frameworks continues to be subjective and problematic. A sequential mixed-methods evaluation design was used to ensure an in-depth evaluation of the specific project objectives listed above. Hence the evaluation is both interpretive and constructionist (Guba & Lincoln, 1989) allowing the researchers to take account of the wider contexts to describe, illuminate and interpret, rather than purely measuring and predicting (Parlett & Hamilton, 1972). As evaluation research is an approach which seeks to establish the value and/or impact of a topic such as a programme, treatment, practice or policy to the recipients of the service/stakeholders (Carnwell, 1997; Davidson, 2005; Ovretveit, 2000), it is distinguishable from other kinds of research by why it is done rather than how it is done (Casswell, 1999). It is a particularly useful form of research as it enables one to determine the value of the topic through the use of inductive and/or deductive approaches (Clifford, 1997), and to uncover the important factors latent in a particular situation (Parlett & Hamilton, 1972). The evaluation approach depends on the type of topic being evaluated and the overall purpose of the evaluation. 3.2 Research design As previously stated the approach to this research was a sequential mixed-methods evaluation and was completed in four phases over an eight-month timeframe. Each sequential phase focused on a particular evaluand grouping which served to inform the basis for the next phase of data collection and analysis. The research design was informed by the work of Parlett and Hamilton (1972), Guba and Lincoln (1989), and the later works of Ovretveit (2000), Davidson (2005), and Miller and Fredericks (2006), all of whom advocate the importance of identifying and seeking the views of different stakeholders (Carnwell, 1997) through the use of multiple data collection strategies. The internal logic of a sequential mixed-methods evaluation design presents methodological strengths (methodological triangulation) not evident in other designs (Miller & Fredericks, 2006; Sandelowski, 2000). As each phase of the research process incorporates its own distinct sampling, data collection and analysis method, which in turn has its own measures of rigour dependent upon the qualitative or quantitative perspective being undertaken, careful application of the sequential mixed-methods design provides a consistent, rigorous and acceptable justification for the research approach (Miller & Fredericks, 2006; Parlett & Hamilton, 1972). 6303_nurse_competency copy copy.indd 24

NURSING COUNCIL OF NEW ZEALAND 25 3.3 Methods 3.3.1 Preliminary work Before commencing phase one of the evaluation, a comprehensive literature review was undertaken. Email and/or telephone contact was made with the other regulatory authorities under the HPCA Act 2003, and with nursing regulatory authorities in Australia, the United States of America, Canada and the United Kingdom (identified in chapter two). 3.3.2 Phase One Method; Systematic document review and policy analysis Objective The objective of phase one was to review the historical development of each aspect of the CCF, to analyse current operational policy and processes including the administrative systems, and to review statistical information with regard to annual practising certificates, recertification audits and competence notifications. Historical review Areas of exploration included historical data relating to the development and implementation of the CCF; relevant legislation; and existing Council policies, procedures and guidelines associated with the CCF, including external stakeholder engagement. Simple descriptive analysis was undertaken. Policy analysis Existing and current Council policies, procedures and guidelines associated with the CCF, including recertification requirements, were analysed using a framework informed by Musick s (1998) structured approach to policy analysis. Statistical analysis Statistical information relating to annual practising certificate applications, recertification audits and competence notifications were reviewed and collated since implementation of the CCF framework in 2004 and the recertification audit process in 2005. Details of the collated findings are reported in chapter four. The findings from this phase of the evaluation contributed to the development of the semi-structured interview questions and web-based survey undertaken in phases two and three of the evaluation. 3.3.3 Phase Two Method; Semi-structured interviews with key stakeholders Semi-structured interviews were undertaken with a purposive sample of 26 key stakeholders derived from the following groups: Nursing Council of New Zealand employees, nurse leaders, i.e. directors of nursing, nurse managers from the public and private sector, heads of schools of nursing, nursing representatives from the Ministry of Health and professional nursing organisations, and nurses who have participated in a recertification audit under the CCF in the past four years. Objective The objective of phase two was to determine from key stakeholders their knowledge of, experience with, satisfaction with, and confidence in the CCF as a measure for safe professional practice as a nurse in New Zealand. This included feedback with regard to CCF processes and procedures, and associated professional, legal and ethical issues in relation to the demonstration of continuing competence and safe practice. 6303_nurse_competency copy copy.indd 25

26 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Interview design Semi-structured interviews with a purposive sample of participants provided the researchers with the ability to select participants with the specialised knowledge and/or experiences to contribute information relevant to the research. Potential participants were contacted initially by an email invitation which included a copy of the research information sheet and a written consent form. Those who indicated they were interested in participating in the research were then contacted by telephone and the interview appointment confirmed. The semi-structured interviews were conducted as scheduled ensuring the following elements were addressed: understanding and experience of the CCF, including the recertification audit processes; indicators and evidence of competence; confidence in the CCF as a measure of safety to practise; contextual relevance, including knowledge of the PDRP process. Any additional information offered was noted and organised into themes. The interviews ranged from 25 60 minutes in duration and each was digitally recorded and transcribed. Confidentiality Participation in the interviews was voluntary. Signed consent was provided by each participant. No identifying information has been used in this report with regard to individual interviewees. Data analysis The transcribed interviews were collated and analysed by the researchers using Thomas (2003) general inductive approach, the purpose being to allow findings to emerge from frequent, dominant or significant themes inherent in raw data, without the restraints imposed by structured methodologies (Thomas, 2003, p. 2). This systematic approach to inductive analysis provided a logical process for independent consistency checks, and allowed the researchers to demonstrate rigour and trustworthiness of the findings which are reported in chapter five. The themes which emerged were then used to inform the development of the webbased survey. 3.3.4 Phase Three Method; Web-based survey of nurses A web-based quantitative survey of a representative sample of nurses, registered with the Council and active in terms of the CCF, was undertaken. The web-based electronic survey of 12% (n = 5,339) of nurses was conducted from 1 16 December 2009. An introductory invitation, which included the URL link to the webbased questionnaire, and the research information sheet including the researcher s contact details, was sent to the individual email addresses of the sample group. For the purpose of this research nurses were defined as Nurse Assistants (NA), Enrolled Nurses (EN) and Registered Nurses (RN), who had applied for a annual practising certificate within the previous four years and who consented to participate in web-based surveys. Nurse Practitioners were excluded from this research as they currently have a different continuing competence process. Objective The objective of phase three was to determine satisfaction with and confidence in the CCF, and to seek feedback from nurses in regard to: their understanding of and/or ability to demonstrate the indicators of competence required by the CCF for safe professional practice; o the self-declaration in terms of the professional, legal and ethical issues, and safe practice o the required hours of clinical practice o the required professional development (continuing education) hours access to/engagement with PDRP programmes; satisfaction with the recertification audit process; 6303_nurse_competency copy copy.indd 26

NURSING COUNCIL OF NEW ZEALAND 27 o o o participation in a recertification audit understanding/satisfaction with the process, documentation and requirements in terms of demonstrating competence role of the peer assessor. Survey design The web-based server and software Zoomerang was used to develop and administer the questionnaire. The themes and questions were developed from the combined findings of the previous phases literature review, document review, and semi-structured interviews with key stakeholders. In addition to demographic data, the questionnaire was based on four main themes: competence (indicators, assessment, processes), annual practising certificate applications, recertification audits, and professional development and recognition programmes. The questionnaire was designed to capture a wide range of information through the use of closed (yes/no) and attitudinal questions, resulting in categorical, nominal and ordinal data. Nominal data was derived through categorisation of dichotomous data into two groupings, for example questions with a response of yes or no. Attitudinal questions were measured using a seven-point Likert scale, which comprised a list of positively and negatively worded statements with which the respondents were asked to indicate their strength of agreement or disagreement. The questionnaire was piloted electronically via Zoomerang with a convenience sample of 14 nurses, before final implementation. This process enabled the questionnaire to be tested and feedback provided. Following the initial pilot, adjustments were made to the format, structure and order of four questions. The questionnaire was then returned to the pilot group. Anomalies in two questions were corrected and the questionnaire was confirmed. To reduce the possibility of multiple or unsolicited responses to the questionnaire the web-link was specifically designed to allow only the original invited recipient to respond. The questionnaire was unable to be forwarded through progressive email links. Confidentiality Participation in the questionnaire was voluntary and anonymous. The electronic email address database was developed and uploaded electronically by Council staff directly onto the Zoomerang web platform. No identifying information of potential participants was provided to the researchers. Data analysis Participant responses were initially collated and analysed via Zoomerang and then imported using the software package SPSS (Statistical Package for Social Sciences) for Windows version 17.0 to enable more extensive analysis. Preliminary analysis was completed with the whole group using basic statistical frequencies, numbers, percentages, means and distribution. The data were further analysed by cross-tabulating data and examining the relationship between variables (scope of practice, employment area and practice setting). Analysis of variance (ANOVA) was used when indicated to provide a deeper understanding of the data within and between the groups. Findings are presented in chapter six. 3.3.5 Phase Four Method; Data triangulation and analysis, discussion and recommendations Bias introduced by the researchers can directly affect the validity and reliability of the research findings (Ovretveit, 2000). To reduce the possibility of this occurring three distinct methods of sequential data collection and analysis have been used in the previous research phases, each with their own measures of validity, reliability, inference and/or transferability. In phase four, methodological triangulation (Carnwell, 1997; Davidson, 2005) has been used to evaluate the CCF and to test the degree of convergence and validity of the research findings. Triangulation of the data from phases one, two and three added to the robustness (Ovretveit, 2000) of the research through a broader range 6303_nurse_competency copy copy.indd 27

28 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK of data and perspectives, which in turn enhanced the construct validity. Discussion of the triangulated findings is presented in descriptive and statistical forms to correspond with the Council s research outcomes. The report concludes with recommendations. 3.4 Ethical approval This research conforms with the guidelines prepared by the New Zealand Health Research Council (2002) for the preparation and undertaking of research involving human subjects and the Australian national statement on ethical conduct in research involving humans (National Health and Medical Research Council, 2007). Ethical approval was granted by the Health Research Council of New Zealand, Health and Disability Multi Region Ethics Committee, reference number: MEC/09/64/EXP, the Eastern Institute of Technology, Hawke s Bay, Research Approvals Committee: Ref - 27/09, and ratified by The University of Sydney, Human Research Ethics Committee: Ref 12618. 3.5 Limitations of the research Evaluation research utilising a sequential mixed-methods design is no different from any other form of research Table 2 Council objectives addressed in the evaluation Phase One Phase Two Phase Three Phase Four Literature Review Document Analysis Purposive interviews key stakeholders Web-based questionnaire Triangulation of data 1. Explore the validity of the stipulated hours of professional development and days / hours of practice over a three year period, as indicators of competence 2. Provide information on the validity of undertaking a random audit of five percent of the nursing workforce to meet recertification requirements 3. Document and track the different forms of written evidence that is currently acceptable to the Council to demonstrate competence X X X X X X X X X X X X 4. Identify issues related to peer assessment of competence. X X X X X 5. Develop a framework to enable the Council to complete a further evaluation in five years time X X in that methodological deficits may exist and are not necessarily obvious until the research is undertaken. However, reliability and validity of results depends on how rigorously sampling, data collecting, data collating and analysis are conducted. The sequential nature of the mixed-methods evaluation design and methodological triangulation of data adds strength to the design and the findings (Miller & Fredericks, 2006). Bias introduced by the researchers can directly affect the validity and reliability of research findings. To reduce this possibility the research was undertaken using three distinct methods of data collection and utilised 6303_nurse_competency copy copy.indd 28

NURSING COUNCIL OF NEW ZEALAND 29 methodological triangulation to elicit a broader understanding of the data and information. The document review and policy analysis was extensive and complex. Some historical documentation may have been missed due to the difficulty in sourcing some documents and the absence of a complete set of chronologically indexed and filed documents. Policy documents and statistical summaries were supplied by Council administrative staff. The web-based questionnaire was developed to elicit feedback with regard to the CCF and recertification audit. Validity of the questionnaire was examined following an initial pilot survey which indicated consistency of respondent replies and accuracy in interpretation of the questions. The questionnaire provided nurses with a safe means of expressing their views anonymously and the range of data received was broad. However, issues relating to self-completion questionnaires must be acknowledged; the perception of the respondent is their perception on the day they complete the questionnaire and may alter over time; the respondent may have adopted what they perceive is a socially acceptable position when responding to the questionnaire; or the questionnaire may have been completed by someone other than the specified recipient. Whilst there is no formal evidence of any of these examples occurring they are raised and discussed in relation to the validity of the questionnaire. 3.6 Conclusion This chapter has provided a detailed overview of the research design and methods. Chapter four will present a summary of phase one of the research, the document review, policy analysis and findings. 6303_nurse_competency copy copy.indd 29

30 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Chapter Four Phase One Results - Document Review and Analysis 4.1 Document review A comprehensive document review and analysis was undertaken by two members of the research team. The focus of this review was to identify the documented process by which the Council developed its Continuing Competence Framework and associated policies. Documents reviewed include: papers and memoranda to Council Council minutes published Council documents. 4.2 Framework for document analysis 4.2.1 Historical review Areas of exploration included historical data relating to the development and implementation of the CCF; relevant legislation; existing Council policies, procedures, and guidelines associated with the CCF, including external stakeholder engagement. Simple descriptive analysis of these was undertaken. 4.2.2 Policy analysis Existing and current Council policies, procedures and guidelines associated with the CCF, including recertification requirements, were analysed using a framework informed by Musick s (1998) structured approach to policy analysis. It is clear that extensive consultation has occurred over a number of years during the development and various iterations of the Continuing Competence Framework between 1997 and 2003. However, it is important to note that none could be implemented until the enactment of the HPCA Act 2003. The following section addresses the historical progress of the development of the continuing competence framework. 4.3 History Chronologically, there is a well documented trail of the development of the Continuing Competence Framework over a number of years. The Council initially signalled this in its inaugural published strategic plan 1 April 1994 31 March 1997. This plan identified a number of critical strategic issues (CSIs), one of which was associated with competency-based practising certificates (CSI 5), need to develop performance-based annual practising certificate renewal criteria (NCNZ, 1994). Additional to the issue of competency-based practising certificates were a number of other CSIs, including the need/opportunity to develop a general set of competencies for registration/enrolment, and the development of post-registration competencies and standards. The CSIs in the 1994-1997 strategic plan were used as the basis for a four-stage project (Chappell, 1995). In August 1996 a discussion paper on performance-based practising certificates was released for consultation. The focus of this paper was stated as... a means to ensure public confidence in the continuing competence of nurses and midwives. A paper reporting on submissions relating to this consultation was presented to the Council at its meeting on 28-29 April 1997. The first document outlining a draft framework was developed for consultation in 1997. A working group on competency-based practising certificates (CBCs) was established in 1997, with representation 6303_nurse_competency copy copy.indd 30

NURSING COUNCIL OF NEW ZEALAND 31 from a range of nurses, including Maori. In September 1998, draft guidelines for competency-based practising certificates were developed for wide consultation, and on 1 April 1999 Guidelines for CBCs were published. In March 2001, Towards a Competency Framework for Nursing was published. Essentially this document describes the components of such a framework in anticipation of impending enabling legislation. In November 2001 Guidelines for Competence Based Practising Certificates for Registered Nurses was published. This document aimed to provide nurses with information about a proposed process for renewal of practising certificates once the legislation was in place. A scoping exercise to find out what professional development activities and credentialing processes existed within the nursing sector was undertaken in 2002. In December 2002 a report to the Council proposed a strategic project to further develop the competency assurance framework. Clinical career pathways, or professional development and recognition programmes (PDRPs) as they are now known, had been progressively introduced by District Health Boards since the late 1980s. They were originally introduced to provide a career structure and recognition for nurses in clinical practice. They usually have four levels of advancing nursing practice: new (novice or beginner), competent, proficient and expert. A nurse can have her level of practice assessed by submitting a portfolio. The New Zealand Nursing Organisation had also introduced certification for nurse specialists and nurse consultants. The Council received feedback on the guideline for competence-based practising certificates that its requirement for a personal professional profile was a duplication of PDRP and professional organisation credentialing requirements. The Council responded by setting up an accreditation process for these programmes so nurses did not have to duplicate portfolio development to meet the competence-based practising certificate requirements (i.e. nurses on PDRPs are exempt from the Council s recertification audit process). In September 2003, a framework for the approval of profession recognition programmes to meet the requirements of competence-based practising certificates was approved by the Council. It was noted that approval of professional development programmes for ensuring competence was a form of delegation of the Council s function in this regard and therefore it would be important to ensure the approval process was rigorous and programmes required regular audit. Council resolved in September 2003 that when developed, the end of the first year of practice competencies would become the competencies for competence-based practising certificates and that overseas nurses and nurses returning to practice be required to meet the competence-based practising certificate requirements before commencing practice. The Council minutes of 1 July 2003 recorded discussion on the competency assurance framework, and the impact of changes outlined in the Health Practitioners Competency Assurance Bill. It was also noted that there was a general misunderstanding of the Council in the sector, and that often Council members needed to explain and define the role. There was a suggestion that a video be developed for educational institutions and DHBs on key issues such as competence-based practising certificates. A memo to Council (January 2004) recommended approval of the draft Review of the Guidelines for Competence- Based Practising Certificates. It proposed the level of assessment required for continuing competence purposes; issues related to the validity of using portfolios for assessment in this context; the resource implications of a random audit of portfolios; and the time and effort required by nurses to meet the competence requirements. The memo also recommended that continuing competence assessment should be at low level, although what this means was not specified. The Review of the Guidelines for Competence-Based Practising Certificates in March 2004 (NCNZ, 2004b), was a consultation document for which submissions closed on 31 May 2004. This is an important document, which was circulated to a wide range of individuals and groups. It contains important appendices: Appendix 1: The current guidelines for competence-based practising certificates. Appendix 2: Continuing competence requirements of other nursing authorities (Table 1 is a comparison of continuing competence requirements of other nursing authorities.) Appendix 3: Continuing competency requirements of other professions. Appendix 4: Continuing practice competencies for the renewal of practising certificates. Of note in Review of guidelines for competency based practising certificates (NCNZ, 2004b) are several 6303_nurse_competency copy copy.indd 31

32 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK important comments in relation to continuing competence requirements for the renewal of practising certificates. These comments have been reproduced here to provide clarity around continuing competence requirements. The KPMG report (2001) on undergraduate nursing education suggested that process of using initial registration competencies as the marker by which registered nurses practice is measured does not take into account the importance of the initial practice experience following graduation nor recognise the development that occurs during the initial professional experience (KPMG, 2001, p. 103). In addition it is noted that feedback from the Draft Framework for the approval of professional recognition programmes to meet the requirements of competence based practising certificates (December 2002) indicated that the Competencies for Entry to the Register of Comprehensive Nurses (2002) are not considered to be the competent level on clinical career paths and other programmes (p. 103). Further, it is noted that the Health Practitioners Competence Assurance Act (2003) does not define competence, but it does define the required standard of competence as the standard of competence reasonably to be expected of a health practitioner s scope of practice (p 12). This suggests more than entry level practice (NCNZ, 2004b, p. 29). In March 2004, the Council also approved for consultation a framework of new competencies for enrolled nurses and registered nurses. This included two consultation documents: Draft entry level competencies for the registration of the nurse (NCNZ, 2004c) and Draft continuing practice competencies for nurses (NCNZ, 2004d). These competencies were expected to be met at the end of an undergraduate programme and at the end of the first year of practice respectively. An analysis of the submissions for the review of guidelines for competence-based practising certificates for nurses (March 2004) is a document which contains the following headings: Introduction; Submissions process; Analysis; Definition of competence; How do we assess competence?; Who requires a practising certificate?; Definitions of practising; Practice hours; Professional development; Personal professional profile or portfolio; Second level nurse; Pathways to competence-based practising certificates; Audit requirement. Appendix 1: list of group submissions. Appendix 2: full table of responses to audit requirements. Most respondents supported professional recognition programmes as a pathway to a competence-based practising certificate. In this document lack of support of the draft continuing practice competencies is identified. In August 2004 the Council resolved that the competencies for entry to the register of nurses should be the competencies for continuing practice requirements as well. A memorandum to the Council on 1 August 2004 appears to be the final document outlining the Continuing Competence requirements. In June 2005 the Council considered a paper outlining the proposed recertification audit process. 4.4 Policy for recertification and policy for CCF In terms of the Council s policy documents, the two associated with this project are the Continuing Competence Framework, and the Recertification Requirements. The former is available on the Council s website, and is a guideline rather than a policy, but there is no indication of the date this was effective from. There is also a document used in-house outlining the requirements of the CCF, in the form of a memo to Council dated August 2004. The recertification audit process policy (GPO 05.03) (August 2006; May 2008) is an in-house document. It is a guideline and procedure document rather than a policy, and is not publicly available. It does not include a purpose statement and focuses solely on procedural aspects of the recertification audit process. Whilst a clear explanation of criteria for exemption from audit and the recertification audit evidential requirements is available to the public on the Council website, there is no purpose statement linking the audit to the CCF and the Council requirements mandated by the HPCA Act. In addition there is no information available to nurses with regard to the recertification process or timelines once the documentation has been submitted to the Council. 6303_nurse_competency copy copy.indd 32

NURSING COUNCIL OF NEW ZEALAND 33 Currently a package of hard copy documentation is posted to individual nurses selected for recertification audit. This package includes: a form letter recertification audit; information sheet why have I been selected for a recertification audit?; nurse audit checklist. Nurses are directed to the Council website to download and print the relevant template documents for example competencies for the scope of practice and competence assessment forms. These documents are not included as hard copy in the recertification audit package and are only available as a PDF template from the website. Hence nurses are required to source the relevant documentation and enter their evidential data by hand rather than being provided with the opportunity to download and enter their data electronically. According to Musick (1998), policy analysis is concerned with two distinct but related processes the contents of the policy and the process by which it was developed. Musick notes that often policy initiatives are advocated or described, without an examination of the process by which they were developed, or who was involved. He provides a useful framework for policy analysis, and this framework has been adapted to inform the analysis of key policy documents associated with the Continuing Competence Framework. The summarised analysis of key Council policy documents associated with the CCF is presented in Table 3. Table 3 Analysis of key Council policy documents associated with the Continuing Competence Framework Components of framework CCF Framework Audit Process (Recertification Policy) Conceptual: What are the core concepts under discussion? How are they defined? What are their measurable outcomes? Competence Continuing competence Competence Continuing competence Recertification Normative: What ought to be true in regard to the policy? Do current views of key people or groups differ? Theoretical: Within what theoretical framework(s) does the policy fit? Empirical: Are there research studies in the literature which could be helpful in illuminating the issues? What important facts can be gleaned from these studies? Economic: What impact would the adoption of the policy have on budgetary resources? What economic structures would need to be in place in order to implement the policy? Cultural: How are different organizational, racial, gender and/or professional cultures affected by the policy? Ideological: How are the ideological and informational aspects of the policy interwoven? Do various parties participating in the policy development process bring strong ideological frameworks into the discussions? Policy developed from a range of submissions from stakeholders Legal/regulatory Literature from other nursing regulatory authorities (UK, Canada, and Australia) has informed the development of the CCF. Some of this is based on empirical research. Recommendation in 1998 that the Council documents a full cost-benefit of the introduction of competence based practising certificates. Working party to develop CBCs included Maori representation. Decisions from submissions to consultation documents generally went with the majority view. Policy developed from a range of submissions from stakeholders Legal/regulatory Literature from other nursing regulatory authorities (UK, Canada, and Australia) has informed this process. Some of this is based on empirical research. Significant impact on budget resources noted if the Council required to audit all nurses mitigated to some extent by the development of a framework for the approval of PDR Programmes (PDRP) Not identified Decisions from submissions to consultation documents generally went with the majority view. Framework informed by Musick (1998) Policy analysis in medical education: A structured approach 6303_nurse_competency copy copy.indd 33

34 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 4.5 Council statistics Figure 1 depicts the number of practising certificates issued by the Council since the enactment of the HPCA Act (2003) and implementation of the CCF (2004). These figures are inclusive of all nursing scopes of practice, new graduate registrations and registration of internationally qualified nurses. Figure 1 Practising Certificates issued As depicted there was a significant reduction in the number of practising certificates issued between 2004/2005 and 2006/2007. From 18 September 2004 Council implemented the CCF and issued interim practising certificates for periods of three, six, nine or 12 months on a quarterly basis to correspond with an applicant s birth date. The numbers for 2004/2005 reflect this initial process. In December 2005 Council implemented the recertification audit process. Nurses who did not meet the Council s requirements for ensuring continued competence were issued with interim practising certificates under section 31 of the HPCA Act 2003. These were replaced with full practising certificates as the nurses met the required conditions. It appears that a number of nurses who were no longer practising, or who no longer met the continuing competence requirements, elected not to reapply for a practising certificate during this initial two year period, as the numbers fall quite steeply. Other possible reasons for the decline in the number of practising certificates issued are that from September 2004, midwives were regulated by the Midwifery Council and many midwives chose not to renew their nursing practising certificate 2 and that the Council applied a definition of practising in August 2004 which stated : All nurses who are working in a capacity for which a nursing qualification is required in order to practise in direct relationship with clients or in nursing management and administration, nursing education nursing research or nursing professional advice or policy development require a practising certificate. Therefore some nurses in non-traditional nursing roles may not have renewed their certificate. This definition remained in place until June 2006 when it was replaced with the more inclusive current definition. Of the 421 nurses selected for audit in the 2005/2006 year, 50 nurses chose not to participate in the recertification audit or renew their practising certificates. From 2006 to 2008 issue of practising certificates remained relatively 2 From 1 April to 17 September 2004 the Council issued 3,547 midwives with annual practising certificates, 1,257 of whom registered as midwives only (NCNZ, 2005). 6303_nurse_competency copy copy.indd 34

NURSING COUNCIL OF NEW ZEALAND 35 stable but in the 2008/2009 year there was a significant increase of 12% recorded. This appears to be due to an increase in registration of new graduate registrations (6%) and of internationally qualified nurses (12%). Figure 2 depicts the recertification audit and competence notification trends over the period since implementation of the CCF. Nurses who are levelled on a Council-approved PDRP are exempt from the recertification audit process. The increase in the number not meeting requirements in 2007/2008 is due to an administrative decision to issue nurses with interim practising certificates with conditions at the time the practising certificates expire rather than to wait for nurses to supply further evidence before issuing them with annual practising certificates. Figure 2 Recertification audit trends and competence notifications Under the HPCA Act 2003 section 34, the Council may review the competence of a nurse if she/he has not maintained the required standard of competence; if there is evidence to suggest the nurse s practice poses a risk of harm to the public; or at any other time. Notifications are made through the following mechanisms: by an employer when a nurse has resigned or been dismissed for reasons relating to competence; by the Health and Disability Commissioner or the Director of Proceedings if he or she believes that a nurse poses a risk of harm by practising below the required standard of competence; and by any health professional who believes there is a competence issue. This is a process independent of the CCF and the competence notification trends are independent of the CCF recertification audit. It is reassuring to note that nurses notified for competence concerns are a very small proportion (0.02%) of the nurses with practising certificates (NCNZ, 2009). 6303_nurse_competency copy copy.indd 35

36 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 4.6 Summary of findings from the document review and policy analysis This section summarises and highlights the findings from the document review, policy analysis and statistical trends with regard to practising certificate renewal and competence notifications. These findings will be discussed in more detail in chapter seven where the results are triangulated. 4.6.1 Development of Continuing Competence Framework Evidence of well researched and detailed development of the CCF which included extensive stakeholder involvement, consultation, and feedback over an eight- year period. Selection of continuing competence indicators self-declaration, practice hours and CPD hours based on best international evidence at the time of implementation. Decision with regard to recency of practice and minimum number of hours required in a three-year period appears pragmatic and was based on what was considered fair and reasonable. Selection of 5% of nurses for recertification audit per year appears to be a decision based on wide consultation and discussion at the time of implementation. There is no documentation that suggests the selection of 5% is based on empirical evidence. 4.6.2 Policy documents CCF policy and recertification audit policy: in-house procedural policy rather than overarching policy documents. 4.6.3 Recertification audit Internal audit of recertification programme completed in 2006. Evidence of review of processes and implementation of quality improvement outcomes as a result of the internal audit. No facility for online submission of audit documentation. No facility for tracking the recertification audit documentation by the applicant post submission. Process for internal moderation of recertification audit documentation ad hoc and requires review to assure inter-rater reliability and transparency of audit process. 4.7 Conclusion The Health Practitioners Competence Assurance Act (HPCA Act) was enacted on 18 September 2003. As noted earlier, it was not until the enactment of this legislation that the regulatory authorities, including the Council, had a legal mandate to provide a mechanism to ensure the competence of nurses. However, the various iterations of guidelines for competence-based practising certificates and other consultative processes clearly demonstrate the Council had prepared the ground well for the implementation of the CCF and developed a comprehensive evidence-based framework. Whilst the findings indicate some areas for attention these will be further discussed in chapter seven in association with the findings from Phase Two (qualitative interviews) and Phase Three (quantitative questionnaire) of the research. 6303_nurse_competency copy copy.indd 36

NURSING COUNCIL OF NEW ZEALAND 37 Chapter Five Phase Two Results - Qualitative Interview Data 5.1 Introduction As previously discussed 26 interviews were undertaken by one team member, with a purposive sample of key stakeholders representative of the following groups: registered nurses who had participated in a recertification audit; nurse managers and directors of nursing from district health boards; primary and private sector organisations including non-governmental organisations (NGOs), elder health and disability services, Ministry of Health, professional organisations and schools of nursing. Previous Chairs of the Council, previous and current Council staff were also interviewed to provide additional data in relation to the historical, developmental and operational aspects of the CCF, and to provide consistency and validation of the findings from the document review and policy analysis completed in phase one. Each interview ranged from 25 60 minutes in duration and was recorded, coded and then transcribed. The transcribed data were analysed using Thomas (2003) general indicative approach for qualitative data analysis. This method was used to condense the extensive and varied raw data into a brief summary format, to establish clear links between the research objectives and the summary findings derived from the raw data, and to allow the researchers to identify categories and themes evident in the raw data. The general inductive approach provided an efficient and systematic model for data analysis. Consistency of data theming was independently checked by another research team member following the initial thematic categorisation and again by the wider team following summary analysis and generation of the sub-themes. The data findings from the interviews are presented in this chapter and were used to inform the development of the web-based questionnaire. Triangulation of overall findings is presented in chapter seven. It is important to note that the overwhelming response by the participants indicated a positive commitment to the CCF process. Further, they reported a comprehensive understanding of the intention of the CCF and the purpose and intentions behind the process were considered imperative and valuable. In particular the participants endorsed the importance of the CCF process in meeting the Council s agenda of public safety and emphasised the value of the CCF process in addressing the responsibility of individual nurses to engage with the process and maintain competence. Findings are presented under the three thematic categories and 11 sub- themes that emerged from the data. These categories and themes represent some identified areas of confusion or concern surrounding the process and delivery of the CCF. 5.2 Thematic category - Competence 5.2.1 Lack of clarity and understanding 5.2.2 Purpose of the continuing competence framework 5.2.3 Continuing competence indicators 5.2.4 Education and continuing competence 5.3 Thematic category - The role of the Council 5.3.1 Legal status of framework (indicators, self declarations) 5.3.2 Responsibility and accountabilities (role of bodies, people) 5.3.3 Communication and consultation 5.4 Thematic category - The recertification audit process 5.4.1 Peer assessment 5.4.2 Audit requirements (documentation, timeframes, guidelines and templates) 5.4.3 Transparency of the audit process 5.4.4 Communication and processes 5.4.5 Professional Development and Recognition Programmes 6303_nurse_competency copy copy.indd 37

38 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 5.2 Competence Competence was a major and underlying theme that emerged from all of the interviews. Whilst it was generally acknowledged that the Council has defined competence and set standards of practice, some felt there was a lack of clarity amongst some nurses that resulted in a level of confusion, particularly with regard to the purpose of the CCF; interpretation and enactment of the competencies for continuing competence; and/or understanding of how to interpret and provide evidence in relation to the indicators of competence. Issues of validity and consistency were also raised in relation to the indicators of competence and their relationship with continuing competence. 5.2.1 Lack of clarity and understanding This theme emerged from a variety of comments made in relation to a general confusion amongst nurses with regard to the concepts of competence and continuing competence : I think many nurses perceive competence in terms of clinical tasks and competencies, not the whole picture. The current competencies seem quite repetitive and the language is complicated so this doesn t help. I don t think the competencies make it clear that nurses should be assessing their ongoing competence where they are now in terms of practice, rather than just being competent, which could be perceived as a minimum level. There was a general view that clear guidelines are needed for nurses to help clarify the concepts of competence and continuing competence, particularly in terms of ensuring nurses can accurately complete a self-assessment and the self-declaration for the renewal of their annual practising certificate. The following quotes reflect some of the comments: Well I know myself the first time I had to fill out that form of course I was going to tick that I m competent. Cos I think I m competent. But interestingly if you re not knowledgeable about what the competencies are then how do you know. So maybe there should be something on the form that shows you what they are. You know, ticking to say you re competent actually doesn t mean Well I think I am competent. Maybe it should be worded, Can you provide evidence that you [continue] to meet the competencies? and as you may be audited. 5.2.2 Purpose of the Continuing Competence Framework The opening statement for each interview was Tell me about your understanding of the CCF. Generally the interview participants demonstrated they clearly understood the purpose and importance of the CCF in terms of the Act requirements and the role of the Council as the regulatory authority. However, some respondents raised the issue of confusion regarding the purpose of the Continuing Competence Framework suggesting that while the Council has a role in ensuring some degree of competence there is a general misunderstanding on behalf of nurses of what the Council s role is in this regard. Several respondents indicated that the Council should be concerned with the minimum standard of competence and that it is the employers role is to drive a performance development culture: I know that there is a set of competencies which are essentially minimum competencies for each of the scopes of practice NP, RN and EN or NA. That those are required by law under the HPCA and that Council has a - various processes for those three groups in terms of determining that members of the profession meet those [minimum] competencies. That, for the RN scope in DHBs, that process is significantly [related] to the PDRP programme and where there s an established PDRP programme Council audits the PDRP programme or authorises the PDRP programme as a proxy for Council s own auditing process for RNs. RNs levelled in a PDRP are required to meet higher levels of competency. 6303_nurse_competency copy copy.indd 38

NURSING COUNCIL OF NEW ZEALAND 39 Another respondent commented: It is how a nurse retains competence on an ongoing basis [this is the point]. Nurses who have achieved a level on a PDRP are excluded from random audit because they have demonstrated they have achieved the required Council competence plus whatever additional ones are required for their level on the PDRP. For others the framework was seen as a positive and necessary process about setting and maintaining standards and the notion of capturing practice development : The continuing competence framework is a process whereby Nursing Council has set some standards for the amount of professional development that nurses have to undertake in order to maintain their APC. Also, the amount of time that they need to be in actual practice in order to maintain their APC. So it s a way of creating a standard which replaces the previous system of having no standard of what people were engaged in. Another respondent expressed the view that while no system can totally ensure competence it is important to have some system to review competence. The notion that the Council s current CCF is just an indicator of competence was also expressed by this respondent:... an audit done by a regulatory body in no way can say... Oh you re competent to practise. A PDRP process I think can because it goes into much more depth. But anything a regulatory authority does can only be an indicator really. Nursing Council should actually clarify the purpose of the process Because it has, it s got confused with PDRP. Without a shadow of a doubt. The idea of it being minimal and they ve - and it s a - just an absolute focus on safety. Public safety as opposed to professional development. Another respondent suggested: Perhaps the Council needs to make it really clear what they want from the framework, do they want to monitor competence to ensure public safety or do they want to monitor competence, ensure public safety and promote professional career development? I think many nurses are confused because they think the Council owns the PDRP process and it all gets tied up with the competence framework they don t understand PDRP belongs to the employers and NZNO. The issue of indicators of competence was also raised as an area that might be better clarified within the competence assessment process. 5.2.3 Continuing competence indicators do they infer competence? Generally the competence indicators were thought to be adequate. However, the notion of hours worked or hours engaged in professional development as a reflection of competence was not without some criticism. Issues identified ranged from the actual number of hours in comparison with other health professions, the determination of the number of hours and elements that one might capture in a portfolio to indicate competence relating to hours. The idea that indicators might actually be a measure of competence was less clear as one respondent reported: The larger question is, are the competencies valid and reliable really? Because that s the thing that you re measuring them against. So that s - if they re not valid and reliable then actually the process is less important. And whilst they are the competencies that we have, there is quite a bit of literature and research around are they - you know are they actually the measures of what makes someone competent? How do we know that? Have we proven that those are the things that we look for? One respondent reported that portfolios are only useful if you can know what should go in them reporting that: You know everyone s decided that the thing we need to do is create evidence for competence... they re not quite sure what evidence and so they dump everything into a portfolio of evidence cos they don t have the ability or energy all the time to discern what is actually needed. 6303_nurse_competency copy copy.indd 39

40 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK While many respondents debated aspects about hours and recency of practice as valid representations of practice the overall sentiment is captured in the following quote: Yes but even so just because somebody has met all of those requirements - the 60 hours etcetera I m not sure you could still stand with your hand on your heart and say that s a guarantee that somebody s competent. It was suggested by a number of the respondents that nurses as health professionals need to engage more in the process that a self-assessment should be a valid and reliable indicator of competence if undertaken in an honest and thorough manner. The following issue was raised: With self-assessment you rely on the honesty and insight of the nurse, who is a health professional, and in most cases that is fine however there is always a very small group who will fall outside what is acceptable... so the trick is how to validate a self-assessment... The question of what constituted education for continuing competence was reported as a central concern for a number of respondents. There was a general feeling that many nurses did not understand what constitutes continuing professional development nor how it translates to evidence of competence. This element will be further discussed under the theme education and continuing competence. 5.2.4 Education and continuing competence Generally respondents indicated that the requirement for continued professional development/education was an appropriate indicator and expectation for continued competence. However, respondents cited there was a range of activities necessary to maintain clinical skills but raised issues about their direct relevance to competence or public safety. For example, IV accreditation or testing, manual handling, fire drills, CPR and infection control updates might all be considered as continuing education, and the question became which of these activities were simply refreshing existing skills, which led to new knowledge and which played any role at all in assuring a nurse was competent? There appears to be several schools of thought around the professional development. And one school of thought is professional development needs to be new knowledge and building new knowledge to show that you re continuing to build knowledge base. There s another school of thought that any course could be perceived as professional development. I think nurses understand how CPR, IV and all those task-based clinical updates contribute to professional development, it s the other stuff that isn t so clear. The problem really is that people are gathering certificates for this and that purely as evidence, but really it s more about what did you get out of it how did... enhance your practice. That s what is generally not done well, it requires you to think about your practice and that has to be a good thing. Several respondents commented that consistent guidelines around expectations of what constitutes continued professional development activities for continuing competence would be beneficial to a number of nurses. The only comment that I would make is that I think that there needs to be a line drawn in what constitutes continuing competence... I know nurses, for example, who will go to the Nursing Council forum because they know that they will get a lot of hours for the Nursing Council forum. And while that s professional update it s not clinical update. I guess that s one of the things that isn t captured very well by Nursing Council is the fact that their professional development contributes to their practice. One respondent was able to differentiate the confusion around educational activities and competence suggesting that the:... requirement is that people are able to reflect on how a professional development activity enhances their ability to do their job, or enhances their registered nurse role. 6303_nurse_competency copy copy.indd 40

NURSING COUNCIL OF NEW ZEALAND 41 Another respondent reported that if you take the perspective that educational activities are those which enhance the effective practitioner, then activities such as volunteering and non-health-related education might be used to: describe that [activity] in terms of augmenting their ability to be an effective practitioner. Respondents broadly reported anxiety amongst nurses about the educational activities that might count in continuing competence, the framework and indicators. They identified that the portfolio and the role of portfolios for recording competence also caused some confusion. I think there s a lot of confusion there. People don t really know what they should be sort of measuring themselves against and what they shouldn t, I don t think it s a straightforward process for them at all and, in fact, I think what s happening is they re tending to find someone else who s been audited before and seeing how they can match. They don t seem to understand that you don t need a whole portfolio of certificates and exemplars what you need is be able to provide concise evidence that you are competent to practise in the role you are in, so you actually need to focus on the actual competencies and how you meet them.... Both the effort to engage with the process and the lack of clarity around competence requirements were reported as possibly impacting on an individual nurse s willingness to remain in the profession. This issue was also related to the requirements of the recertification audit process. As two respondents commented: I also think that we ve lost - I mean, with the HPCA we ve lost health professionals... they ve said Oh I can t, you know, this is far too hard... There s been a reaction to that and we have lost some good people as a result of it. I mean having your competence questioned and going through a formal process is extremely stressful. I know the legislation wants to be helpful and supportive and that s Council s policy too but it is extremely stressful. I do think that the Council in its notification to the nurse that a competence question has been raised with the nurse - and I know it s probably not the Council s job to do this but the most difficult thing is if the nurse is so upset and then resigns her position because then they can t find another place generally speaking to meet the practice requirement conditions Council puts on them. No evidential detail was provided to substantiate this assertion of resigning from the profession but the perception of the amount of effort required to maintain a record of competence was often mentioned. One respondent, however, commented: I thought I would leave if I was ever audited. But I m a good nurse, so I thought why should I leave... that was my motivator. The other frequently mentioned element was the concern about the effort, the anxiety invoked by the process and the lack of clarity around responsibility for maintaining competence. They don t realise that it s their individual responsibility. And if you work with nurses who have got issues around competence or issues or even being audited. They don t understand that it s their responsibility. They think it s the employer s or they think Well you haven t done this for me. This issue is further reported in the consideration around accountabilities and responsibilities. 6303_nurse_competency copy copy.indd 41

42 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 5.3 The role of the Council The role of the Council in terms of ensuring public safety was discussed by most respondents. Issues with regard to nurses general understanding of the Council s role, jurisdiction and discretion were raised in relation to the CCF. 5.3.1 Legal status of framework (indicators, self-declarations) The Application for Practising Certificate form requires that nurses make a self-declaration that they meet the requirements stipulated by Council. Respondents identified an apparent lack of understanding amongst some nurses with regard to the status of the self-declaration or that the application for practising certificates had in fact changed as a result of the CCF and was now competence based. One commented: I [suspect] many nurses just get the form, tick the boxes, pay the money and hope they will never be audited. I don t think they actually stop and think about am I competent or what they re signing. That probably only happens if they get called for audit. Another commented: Actually... have had the odd case where nurses tick all the boxes and know they haven t done the number of required hours. Other comments reported the reaction of nurses to receiving the package advising that they were being audited. But surprisingly even people who should have an understanding of what it all means often get hooked into providing the evidence. And I wonder if that s a response to the anxiety about they think I m incompetent, so I need to show them everything I ve got that proves that I have a shred of competence. 5.3.2 Responsibility and accountabilities (role of bodies, people) This sub-theme identified a view that nurses may not have a clear understanding about the role of the Council and their personal responsibility for their own competence. There was a recurrent theme that nurses did not understand that this was a part of their professional responsibility and only worried about it if they were audited. One respondent expressed the view that the audit process provided a good wake-up call for these nurses. Other respondents commented:... that reflects a general misunderstanding on behalf of nurses of what Nursing Council s role is. Like Nursing Council couldn t give a toss if a registered nurse is competent or expert. What they need to know is they re safe. If you are a registered nurse my view you have obviously the obligation as a professional to increase your body of knowledge, be able to demonstrate that you re competent at all times. The Nursing Council as a regulatory authority has a responsibility to set the standards, to be clear about the process. But then the health professional has a responsibility to be aware of those standards and to try and meet them. They don t realise that it s their individual responsibility. And if you work with nurses who have got issues around competence or issues, you know that - or even being audited. They don t understand that it s their responsibility. They think it s the employer s or they think Well you haven t done this for me. 6303_nurse_competency copy copy.indd 42

NURSING COUNCIL OF NEW ZEALAND 43 5.3.3 Communication and consultation This lack of awareness of the nurses own responsibilities for maintaining competence was linked to the need for the Council to have good communication and consultation mechanisms in place. A number of respondents commented on the extensive consultation with the profession, undertaken by the Council during the development and implementation of the CCF, and the communication with the nursing sector following implementation, through Nursing Council Forums, website information and newsletters to individual nurses. Despite these communications it was generally felt that a number of nurses still did not have a clear understanding of the role and function of the Council, the CCF process or the responsibilities inherent in being registered as a nurse. The complexity of the Council form for application and renewal of practising certificates (Application for Practising Certificate) was commented on by several respondents. They described the form as being difficult to follow and cluttered. One respondent reported: It s an unmitigated disaster it s very cluttered and it doesn t flow very well. It is difficult to find information about competence until you get to the back page. Another respondent indicated they did not know the Scopes of Practice and associated competencies were on the form at all. One of the difficulties is there isn t any information about the competencies so how do you know how to assess them. 5.4 Recertification audit process This theme was specifically related to discussion on the recertification audit process. However, as will be noted, comments made by some respondents also overlapped with other categories and themes. 5.4.1 Peer assessment In relation to peer assessment for validation of competence to practise, there was a range of issues reported. The question of who should be a peer assessor was raised on a number of occasions. This included opposing views that on the one hand the peer assessment must always be undertaken by a nurse and on the other that in some situations it may be appropriate that another health practitioner who works closely with a nurse could undertake peer assessment. In both situations it was agreed there needed to be clear and explicit assessment criteria. I think it has to be a nurse. And I just think it s all a part of the development of a professional attitude and recognition of your particular skills and knowledge... so if I m the nurse being audited and I go down the road to meet someone at an Iwi provider and say...can I talk to you about how I meet my competencies... and then of course the Council need to know about the person that s signing you off. I can t see any reason why a health practitioner who works closely with the nurse cannot be a peer assessor, so long as they have the right criteria and tool to assess the person against. Let s be realistic, nurses work in a wide range of settings and sometimes the team they work with doesn t include another nurse. I might be an OT or doctor or some other health professional... that person probably knows more about the nurse s practice... is better able to comment than someone they don t work with. I know it s about being a nurse but surely if the criteria are clear, I suppose I mean the competencies, it should be fine. The validity of the peer assessment process was also discussed at length, particularly the issue of who should be a peer assessor and the criteria for their selection. The following comments were made: I do know that some registered nurses don t engage formally in a through the formal processes in a DHB and rely very heavily on peer review to support their practice. And that s the friends thing. 6303_nurse_competency copy copy.indd 43

44 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK I don t think the process is adequate because it is about peer. And it s about choice. And that s why I come back to saying let s get real about this and have a process which is visible, which is mandatory. So we can say to the public that all our nurses that are engaged in registered nursing duties for the public have met a competency framework that is visible....well it s hard to know if they re valid because people will interpret what evidence is required for them the best way they can. I think there could be more guidelines. Peer assessments I think that - that in itself lends it a whole new perspective on people who might sign off that somebody else is competent because they are - they need the staff. And they also - I mean I suppose I could say I know of some cases where people have signed off people as being competent with a peer review, or validated stuff when it s been thrust under their nose. We rely on the nurse to choose a peer assessor, we rely on the peer assessor to be honest and base their assessment on evidence in an ideal world this is fine, but we know there will always be outliers and somehow we need to be sure that the peer assessment is valid based on evidence this is about public safety. It s good it s a reasonable expectation to use peer assessment but surely we can tighten up the loop. 5.4.2 Audit requirements (documentation, timeframes, guides and templates) The audit process itself raised a number of concerns and misunderstandings. While respondents indicated generally that they understood that there were several requirements to be met in relation to the audit process, what was of most concern was the limited access to the appropriate forms. Participants reported they were not supplied with hard copy forms but were directed to the Council website to print off forms. Several participants commented they found this process frustrating and in addition: There was no hard copy of the competence forms I got one off the website, but couldn t type into it. Another respondent commented: You know again I think things are complicated... I m talking generally, not just with the form. I think there s an awful lot of surveillance with nurses that s unnecessary and just over-reactive... In terms of timeframes and tracking of documents after the audit documentation is submitted, concern was expressed by two respondents. One commented: It [submitted audit documents] just went into a deep hole... I rang the Council to say I didn t have a practising certificate yet, and was told not to worry. But I did worry. My employer worried about liability. And another two respondents commented: I got my documents in within the timeframe, but I have no idea what had happened to them after that. I heard nothing until my APC arrived in the mail about 10 weeks later. I have conversations with nurses who have been in audit. Some of them are really positive. Feeling that they ve actually been made to stand back and think about their practice and get peer reviews, and some of them of course are stressed out. Not having clear guidelines was reported by a number of respondents. This issue appeared to be specifically in relation to completion of the self-assessment and peer assessment documents. Two respondents commented that: So rather than you re competent and we re not expecting anything else it s more of a feeling that I m incompetent and I need to prove that I am competent. Which creates anxiety for people. And as they go through the process I think the - as the years have gone by there s been a streamlining of the evidence that s required and also I think nurses have got a bit better at understanding what a portfolio of evidence is, rather than the shopping trolley. 6303_nurse_competency copy copy.indd 44

NURSING COUNCIL OF NEW ZEALAND 45 I know there was information on who to contact if I needed guidance but don t you think if the competencies were written in clear language and less repetitive that people wouldn t need to ask for guidance. It s OK for people who write this stuff all the time but I m a practitioner and really I just want to know what evidence they want for each criteria and then I know what to provide. I suppose what I sent must have been OK because I got a practising certificate I didn t get any feedback about my documents but I did get a letter to tell me in future to make sure my peer assessor wrote more comments I thought that was a bit punitive. Several respondents commented on the recertification audit information and documentation they received. The respondent comment below is most representative of the wider discussion: I got the package from Nursing Council and thought oh then I thought well, get on with it. But it didn t have any forms or anything [it] just told me to get them from the website, initially they weren t all available I think the competency form wasn t there, anyway I got one from someone else who had been through it, but you couldn t type straight on you had to hand write. I don t really have a problem with the process [being audited] because I felt good afterwards I had evidence and proved I was competent, but I felt the documentation you receive should be complete, that would have made the process a lot easier and less stressful Elements expressed in this sub-theme overlap with elements which emerged in the following two sub-themes. 5.4.3 Transparency of the audit process Concerns were raised by respondents about the transparency of the audit process in terms of who would be assessing the material provided and what criteria would the material provided be assessed against. Confidentiality was another concern. One respondent commented: There was no indication of how it would be assessed or who would assess it... [it] really concerned me that I had no idea of who was going to be viewing this information... So I did worry about the notion of confidentiality. One respondent was concerned about being identifiable, and reported It wasn t an objective - well it wasn t an assessment that was any nurse making application, it was because I was identifiable. The comments were specific because they knew who I - what my role was. 5.4.4 Communication and processes Respondents commented on a number of issues in relation to communication and processes associated with submission of evidence for recertification. Other comments related to tracking the audit process and feedback with regard to the appropriateness and acceptability of the evidence submitted. I just received the Annual Practising Certificate... we only hear back from Nursing Council when there s an issue with a nurse s submission and that has happened on two occasions with nurses who have had to resubmit because it has been deemed that they haven t provided enough evidence for particular criteria. It seems the only feedback that you do get is by way of... the nurse being rubber stamped with their practising certificate. I would have liked some feedback, because you put a lot of effort in and in my situation I was working blind, no-one else had been audited it just seemed to go into a dark hole and I heard nothing, then one day my practising certificate turned up in the mail. I presume everything was OK. 6303_nurse_competency copy copy.indd 45

46 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK The public safety focus was the focus of my audit information which was my - about my clinical practice. And I provided not only a verified list of education but also copies of the certificates that were relevant to my clinical practice. And in my appraisal from the senior nurse it was clearly identified that I had completed a post-graduate cert in and participated in the compulsory education within the workplace. I also had my education verified by another senior nurse. And yet that was questioned. Now three different pieces of evidence around my professional development which well exceeded the hours required and I was asked to provide more information. In addition the need for electronic submission and access was also raised. We live in an electronic age so why isn t everything available to be done online. It would be a lot easier to fill in the forms and to send them to your assessors, rather than printing everything off providing it s actually available of course. We can look up nurses current registration online, so surely something could be set up for tracking our audit documents. It might take away the mystery and anxiety if you knew where your documents were and you could say to your employer well I submitted them and at this stage Generally there was a feeling that the recertification process was reasonable. However, the availability and access to guidelines, required assessment forms and tracking of audit documentation could be improved. Communication in terms of feedback from the recertification audit was raised as an important area for consideration by some respondents; but others posed the view that feedback is not the role of the regulatory authority which should only be concerned with ensuring public safety and administering associated monitoring processes. 5.4.5 Professional Development and Recognition Programmes (PDRPs) PDRPs were identified as a sub-theme in relation to recertification and audit requirements because a number of respondents referred to confusion about the role of the Council and PDRPs. There was a sense that Council had muddied the waters by setting the criteria for, and approving, PDRPs. I think people think that the PDRP programme is the work of the Council. When in fact it is about professional development and career development not competence. That s why I don t think the Council should be involved in approving them. Other than - the competence levels fine. So they should only be interested in the programme in the terms of if you have a professional development programme and it meets the minimum competence for continuing competence as per the Nursing Council Framework then your staff that are engaged in that programme can be excluded from audit. I think part of the confusion is because the Council has got guidelines for PDRP programmes but it also approves them... I think the Council dabbled in something that it really shouldn t have been doing in terms of those PDRP programmes approval when the focus was not on whether the people - the expert and proficient levels - but that clearly the competence - the competent level was the key point.... had to keep saying to them, no under the Act you do not actually have to be part of a PDRP process because the Act and Council did not require that. The employers could choose to make that a mandatory part of their employment but actually legally under the Act it was not a mandatory part of the process. [What is]... wanted is evidence. And so I used to say to the staff as long as you can provide evidence and you can actually address those competencies and how you meet them. And you can be really clear that your evidence meets them then there s no reason why you won t gain registration. However not all respondents considered the use of PDRPs as a means of providing evidence of continuing competence to be problematic, as can be seen by the quotes below. So it s kind of been two-fold I guess. On one hand I know that where the responsibility for the Nursing Council competencies comes in and we have an employer driven PDRP programme. But the two of 6303_nurse_competency copy copy.indd 46

NURSING COUNCIL OF NEW ZEALAND 47 them actually, I think, are linking in really nicely now and giving clarity about what the expectations are in one and the other. They re being /if - when - now that they re keeping their PDRPs up to date it s not nearly so difficult to translate when they are audited now you know, how those standards are met - how they re meeting those standards, what - you know, what evidence they can give because they re starting to build that up within their PDRPs. what I m saying about the PDRP thing I think that s raised the whole profile because they know they re not going to get audited and so therefore if they talk to their colleagues. Issues were raised with regard to transferability between programme providers even though the programmes are all approved by the Council. Some were of the view that Council should take a more proactive role in PDRPs to ensure standardisation and transferability. I think that even though the Nursing Council approves PDRP to a common standard it s... It s pretty obvious that there s an inconsistency in what the PDRPs actually relate to. Particularly, in terms of their transferability from one employer to another... I think there needs to be more standardisation of the PDRP...we do need a stronger steer from Council in terms of standardising - towards a national PDRP process. There were also mixed views expressed about whether PDRPs should be mandatory or voluntary. Some felt that PDRPs were such a good idea there was no reason why they should not be mandatory and that there would be something wrong with people who did not wish to belong. And I can t see why... why being on a PDRP programme is something that you wouldn t want to do. So the notion of it being voluntary or not voluntary is the case. But really, I would think that nurses who don t want to be on a PDRP programme should be asking why that is. You know, why would I not want to be engaging with my colleagues. In terms of helping me determine my competence and my fitness to practise and my career progression. If I was thinking that, I would be quite concerned. I react to compulsory anything. But I can see that if I was a leader of a DHB that I would want as many as possible in that process. To show them well I know that my workforce is working to a certain standard. And I would really be concerned with those that aren t in the process. Especially as usually... it s the good people that are going on these processes and it s the people that are out the outliers that are actually quite often the trouble, not function quite as well... So yeah, I wouldn t support compulsory, but I would say strongly encourage people to be on it. 5.5 Summary of findings from the interviews A summary of the findings from the interviews is presented below. These findings will be discussed in more detail in chapter seven where the results are triangulated. 5.5.1 Competence Consensus view that the CCF is an important process for ensuring continuing competence of nurses and public safety. Lack of clarity and understanding about the concepts of competence and continuing competence. Lack of clarity in relation to assessment of continuing competence, confusion over notion of minimum standard of assessment. Consensus that the indicators of competence are appropriate. However, the stipulation of a minimum number of hours is questioned in terms of ability to provide a valid inference of competence. Confusion over what constitutes educational or continuing professional development activities for continuing competence. 6303_nurse_competency copy copy.indd 47

48 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 5.5.2 Role of the Council Extensive consultation with the nursing profession over the development and implementation of the CCF. Confusion regarding the purpose of the CCF and the role of the Council in ensuring competence and/ or continuing professional development. Perceived lack of clarity with regard to the responsibilities of Council versus the responsibility of the employer. Legal status of the self-declaration (Application for practising certificate form) questioned. No explanation on the form in relation to status of the declaration. Issues were raised with regard to verification and validity of the self declaration made on the Application for practising certificate. Application for practising certificate form considered difficult to follow (cluttered) not user- friendly. An identified need for clear guidelines of what constitutes evidence of continuing competence. Perceived lack of clarity amongst nurses with regard to who is responsible for the PDRP process. 5.5.3 Recertification audit Recertification audits are considered a useful tool for the promotion of professional responsibility and accountability, and validation of the CCF. Issues were raised in relation to the validity and reliability of the self-assessment and peer assessment processes. Reported repetition and complex language used on the competence assessment forms. Development of clear guideline documents in relation to what constitutes evidence of continued competence to practise and continued professional development. Reported difficulties accessing assessment forms and entering data directly. Suggestion made that a full set of audit documentation is provided to each participant with the option of downloading electronic documents. Provide access to electronic documents that allow direct data entry. Investigate system for electronic submission and tracking of audit documentation. 5.6 Conclusion This chapter has summarised and presented the themes and sub-themes that emerged from the interviews with key stakeholder respondents. The findings include process and delivery issues, and concerns regarding clarity and use of the applications, recognition of competence activities and the need to validate some indicators. The respondents reported strong support for the CCF as a mechanism and were supportive of the influence of the CCF to improve nurse accountability. The reported confusion regarding the expression of competence is consistent with a contextual and subjective stance towards understanding the concept. These findings informed the development of the web-based questionnaire, the results of which are presented in chapter six, and contribute to the data triangulation and discussion presented in chapter seven. 6303_nurse_competency copy copy.indd 48

NURSING COUNCIL OF NEW ZEALAND 49 Chapter Six Phase Three Results - Quantitative Questionnaire Data 6.1 Introduction As described in chapter two, phase three of this research focused on collection of data from nurses via a web-based (Zoomerang) questionnaire. Essentially the questionnaire was designed to elicit demographic data in addition to information on four key themes (Competence and fitness to practise, Peer assessment, Recertification audit, PDRP), which were drawn from the findings of the previous research phases. The data were coded and collated independently via the Zoomerang software and server, then further analysed using the Statistical Package for Social Sciences (SPSS) for Windows version 17.0. Any errors or inconsistencies in data were carefully screened out by evaluating the range of values generated by running the descriptive frequencies. Statistical results are presented in this chapter in the following sections: 6.2 Demographic data 6.3 Competence and fitness to practise 6.4 Recertification Audit 6.5 Professional Development and Recognition Programmes 6.2 Demographic data 6.2.1 Distribution and return of the research questionnaire An email invitation to participate including the URL link to the web-based research questionnaire was distributed to approximately 12% (n = 5,339) nurses registered with the Council and active in terms of the CCF since 2005. Of the 5,339 emailed invitations to participate, 1,764 were not viewed or responded to, 800 invitations were hard bounced - marked as undeliverable, and 461 participants elected not to complete the questionnaire by submitting an opt-out response. Two thousand three hundred and fourteen (2,314) potential participants viewed the questionnaire and 1,157 submitted completed questionnaires. The questionnaire link was active for a two week period from 2 16 December 2009, during which time access was only available to the participants. Data relating to participation and response rates (completed questionnaires) is presented in Table 4. Table 4 Participation rates and sample size Sample Size Participation Rate Population Size Margin of Error Confidence Level Response Distribution 1,157 50% 45,000 2.85% 95% 50% The data were collated by the overall response, and further analysed using the variables, scope of practice and practice area. 6.2.2 Scope of Practice Of the total respondent group (n = 1,157), 1% (n = 7) identified their scope of practice as Nurse Assistant, 4% (n = 45) as Enrolled Nurse, and 96% (n = 1,105) as Registered Nurse. As reflected in Figure 3 the respondent sample is representative of the overall population of nurses who held current practising certificates as at 31 December 2009. 6303_nurse_competency copy copy.indd 49

50 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Figure 3 Representation of questionnaire respondents A cross-tabulation of the overall group response to the question what is your highest qualification? by the variable scope of practice is presented in Table 5. Table 5 Cross tabulation highest qualification by scope of practice Hospital Certificate Graduate Certificate Graduate Diploma Bachelors Degree Postgraduate Certificate Postgraduate Diploma Masters Degree PHD Other, please specify Total What is your scope of practice? Nurse Assistant Enrolled Nurse Registered Nurse 1,157 7 45 1,105 177 1 30 146 15.30% 14.30% 66.70% 13.20% 34 2 8 24 2.90% 28.60% 17.80% 2.20% 168 0 0 168 14.50% 0.00% 0.00% 15.20% 387 1 2 384 33.40% 14.30% 4.40% 34.80% 158 1 0 157 17.70% 14.30% 0.00% 14.20% 106 0 0 106 9.20% 0.00% 0.00% 9.60% 74 0 0 74 6.40% 0.00% 0.00% 6.70% 11 0 0 11 1.00% 0.00% 0.00% 1.00% 42 2 5 35 3.60% 28.60% 11.10% 3.20% 6303_nurse_competency copy copy.indd 50

NURSING COUNCIL OF NEW ZEALAND 51 A compilation of the overall group response is depicted in Figure 4. Figure 4 What is your highest qualification? - overall group response Two respondents identified they did not hold a current practising certificate and 72 respondents (6%) were not currently employed as a nurse. 6.2.3 Employment setting and practice area A collation of current employment settings of the respondents is depicted in Figure 5 and reflects the diversity and range of employment settings across the respondent group. Whilst the greater percentage of respondents 58% (n = 642) indicated they were employed by District Health Boards (DHBs), 42% (n = 503) indicated they worked in a variety of private, government and non-government agencies. 6303_nurse_competency copy copy.indd 51

52 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Figure 5 Current employment setting - overall group response A cross-tabulation of respondents current employment setting by their scope of practice is presented in Table 6 and indicates the diverse demographic spread of the overall respondent group. Fifty-six per cent indicated they were employed by District Health Boards (DHB) and 44% indicated they were employed across the range of private, NGO, PHO, Maori Health, rural health, education, management, elder health, government and other agencies. 6303_nurse_competency copy copy.indd 52

NURSING COUNCIL OF NEW ZEALAND 53 Table 6 Cross tabulation of current employment setting by scope of practice Total Nurse Assistant What is your scope of practice? Enrolled Nurse Registered Nurse 1,145 6 44 1,095 DBH (Acute) DBH (Primary Health/Community) DHB (Other) Private Hospital Primary health (NGO / PHO) PHO Aged Care Sector (Rest home / Residential Care) Nursing Agency Self Employed Maori Health Service Provider Rural Health Management Educational Institution Government Agency (MOH, ACC, Corrections Service, Defense Forces) Other please specify 460 2 6 452 40.25% 33.30% 13.60% 41.30% 98 0 6 92 8.60% 0.00% 13.60% 8.40% 84 0 6 78 7.30% 0.00% 13.60% 7.10% 86 0 2 84 7.50% 0.00% 4.50% 7.70% 81 0 2 79 7.10% 0.00% 4.50% 7.20% 22 0 0 22 1.90% 0.00% 0.00% 2.00% 101 3 14 84 8.80% 50.00% 31.80% 7.70% 14 0 0 14 1.20% 0.00% 0.00% 1.30% 13 0 0 13 1.10% 0.00% 0.00% 1.20% 9 0 0 9 0.80% 0.00% 0.00% 0.80% 8 0 0 8 0.70% 0.00% 0.00% 0.70% 15 0 0 15 1.30% 0.00% 0.00% 1.40% 29 0 1 28 2.50% 0.00% 2.30% 2.60% 18 0 0 18 1.60% 0.00% 0.00% 1.60% 107 1 7 99 9.30% 16.70% 15.90% 9.00% 6303_nurse_competency copy copy.indd 53

54 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Figure 6 depicts the current practice areas represented by the overall respondent group. Twenty-nine different employment areas were identified by respondents. By far the largest group of respondents were those who identified as practising in medical (n = 138) and surgical (n = 151) services, with perioperative care (operating theatre, n = 89) and continuing care (elder health, n = 82) being the next largest respondent groups. Figure 6 Current nursing practice area overall respondent group A cross-tabulation for current area of nursing practice by scope of practice is presented in Table 7 and demonstrates the demographic distribution of respondents by number and percentage distribution. 6303_nurse_competency copy copy.indd 54

NURSING COUNCIL OF NEW ZEALAND 55 Table 7 Cross tabulation current area of nursing practice by scope of practice Total Nurse Assistant Enrolled Nurse Registered Nurse Emergency and Trauma Intensive Care/Cardiac Care Peri Operative Care (Operating Theatre) Surgical Medical Palliative Care Obstetrics/Maternity Child Health, including Neonatology School Health Youth Health Family Planning/Sexual Health District Nursing Practice Nursing Occupational Health Primary Health Care Public Health Continuing Care (Elderly) Assessment and Rehabilitation Mental Health (inpatient) Mental Health (community) Addiction Services Intellectually Disabled Nursing Administration and Management Nursing Education Nursing Professional Advice/Policy Development Nursing Research Non-nursing health related management or administration Other non-nursing paid employment Not in paid employment 1,157 7 45 1,105 60 0 0 60 5.20% 0.00% 0.00% 5.40% 58 0 0 58 5.00% 0.00% 0.00% 5.20% 89 0 1 88 7.70% 0.00% 2.20% 8.00% 151 0 3 148 13.10% 0.00% 6.70% 13.40% 138 1 4 133 11.90% 14.30% 8.90% 12.00% 25 0 2 23 2.20% 0.00% 4.40% 2.10% 16 0 1 15 1.40% 0.00% 2.20% 1.40% 40 0 1 39 3.50% 0.00% 2.20% 3.50% 3 0 1 2 0.30% 0.00% 2.20% 0.20% 6 0 0 6 0.50% 0.00% 0.00% 0.50% 6 0 0 6 0.50% 0.00% 0.00% 0.50% 33 0 0 33 2.90% 0.00% 0.00% 3.00% 58 0 3 55 5.00% 0.00% 6.70% 5.00% 9 0 0 9 0.80% 0.00% 0.00% 0.80% 54 0 1 53 4.70% 0.00% 2.20% 4.80% 13 0 0 13 1.10% 0.00% 0.00% 1.20% 82 3 14 65 7.10% 42.90% 31.10% 5.90% 47 1 8 38 4.10% 14.30% 17.80% 3.40% 42 1 0 41 3.60% 14.30% 0.00% 3.70% 38 0 2 36 3.30% 0.00% 4.40% 3.30% 3 0 0 3 0.30% 0.00% 0.00% 0.30% 7 0 2 5 0.60% 0.00% 4.40% 0.50% 62 0 1 61 5.40% 0.00% 2.20% 5.50% 46 0 0 46 4.00% 0.00% 0.00% 4.20% 18 0 0 18 1.60% 0.00% 0.00% 1.60% 11 0 0 11 1.00% 0.00% 0.00% 1.00% 13 0 0 13 1.10% 0.00% 0.00% 1.20% 9 0 0 9 0.80% 0.00% 0.00% 0.80% 20 1 1 18 1.70% 14.30% 2.20% 1.60% 6303_nurse_competency copy copy.indd 55

56 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 6.3 Competence and fitness to practise The Council CCF includes three indicators of competence: A. Self-declaration of competence to practise (based on self-appraisal using the Council competencies for the relevant scope of practice); B. Verification of practice hours (minimum of 450 hours/60 days in past three years); C. Verification of professional development (minimum of 60 hours in past three years). Respondents were asked to rank the indicators from 1 (Best) to 7 (Worst) to indicate which they believed provided the best evidence of continuing competence to practise. The indicator of competence to practise ranked best by respondents (n = 470) was the combination of the self-declaration (A), evidence of practice hours (B) and evidence of ongoing professional development (C), which represented 52% of the total overall responses. The least popular indicator of competence to practise was (A) self-declaration only, which represented a response from (n = 311) 40% of the total respondent group. Overall respondent rankings are depicted in Figure 7. Figure 7 Indicators that provide the best evidence of competence to practise 6303_nurse_competency copy copy.indd 56

NURSING COUNCIL OF NEW ZEALAND 57 Respondents were then asked to rank the same indicators from 1 (Best) to 7 (Worst) to indicate which they believed provided the best evidence of continuing professional development. The indicator of continuing professional development ranked best by respondents (n = 402) was the combination of the self-declaration (A), evidence of practice hours (B) and evidence of ongoing professional development (C), which represented 48% of the total overall responses. The least popular indicator of competence to practise was (A) self-declaration only, which represented a response from (n = 322) 43% of the total respondent group. Overall respondent rankings are depicted in Figure 8. Figure 8 Indicators that provide the best evidence of continuing professional development It is interesting to note that for both questions respondents ranked a combination of the three indicators (A, B & C) as providing the best evidence of competence to practise and ongoing professional development. However, it is of note that the respondents ranked the self-declaration of competence (based on self-appraisal of their competence using the Council competencies for their scope of practice) as the worst indicator for providing evidence of competence to practise when used independently. 6303_nurse_competency copy copy.indd 57

58 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Figure 9 The CCF provides a mechanism to ensure nurses are competent and fit to practise In response to the question do you think that the current NCNZ Continuing Competence Framework and processes for renewing practising certificates, provides the mechanism to ensure that nurses are competent and fit to practise? 76% (n = 876) of respondents responded yes and 24% (n = 281) responded no (Figure 9). This response is representative of the nursing population with a margin of error of 2.85% and confidence level of 95%. Responsibility for maintaining continuing competence to practise was a recurring theme throughout the phase two interviews. Questionnaire respondents were asked to rate their level of agreement with four statements (Figure 10). Figure 10 Responsibility for maintaining continuing competence to practise 6303_nurse_competency copy copy.indd 58

NURSING COUNCIL OF NEW ZEALAND 59 Respondents indicated on a seven-point Likert scale (the score of 1 indicated they strongly agreed and the score of 7 indicated they strongly disagreed with the statement). An independent t-test was used to collate the responses to the four statements. Table 8 depicts the collation of responses. Table 8 Responsibility for maintaining continuing competence to practise 1 2 3 4 5 6 7 Overall Respondent Group Strongly agree Strongly disagree Mean Standard Deviation 1. As a health professional I am responsible for maintaining my own competence to practise. n 812 226 74 20 10 5 10 % 70% 20% 6% 2% 1% 0% 1% 1.48.965 2. My employer is responsible for maintaining my competence to practise. n 187 267 252 174 109 85 83 % 16% 23% 22% 15% 9% 7% 7% 3.29 1.769 3. The Nursing Council of New Zealand is responsible for maintaining my competence to practise. n 145 151 174 203 133 158 193 % 13% 13% 15% 18% 11% 14% 17% 4.10 1.989 4. When completing my NCNZ application to renew my practising certificate I understand that I am signing a legal declaration. n 1016 89 23 6 7 3 13 % 88% 8% 2% 1% 1% 0% 1% 1.20.845 *Total respondents N = 1157, 95% Confidence Interval Descriptive statistics were used to calculate the mean (M) scores and standard deviation (SD) for each of the four items. There was no significant variation in responses between the three scopes of practice (NA, EN, and RN) or employment setting. The overall respondent group mean scores for items 1 and 4 indicated there was strong agreement that the individual nurse is responsible for maintaining their own competence to practise (M = 1.48) and understanding that when signing the self-declaration they are signing a legal declaration (M = 1.20). However, for items 2 (employer is responsible) and 3 (Council is responsible) the mean scores were (M = 3.29; SD = 1.769) and (M = 4.10; SD = 1.989) respectively, indicating some ambivalence within the overall respondent group. The standard deviations for items 2 and 3 demonstrate the broad distribution of responses across the seven-point continuum (strongly agree strongly disagree). When items 2 and 3 were cross-tabulated by employment area, findings indicated that nurses employed in Health Management (n = 15) and Educational Institutions (n = 29) scored the highest mean scores for both items. Item 2, My employer is responsible for maintaining my competence to practise (Health Management, M = 4.87, SD = 2.200; Educational Institution, M = 4.07, SD = 1.624) and item 3, The Nursing Council of New Zealand is responsible for maintaining my competence to practise (Health Management, M = 5.33, SD = 1.952; Educational Institution, M = 5.48, SD = 1.661). In phases one and two of this research, misinterpretation of the intent/meaning of the self- declaration questions listed on the Council s Application for Practising Certificate form was introduced as a possible issue in terms of the following: nurses not completing the documentation accurately; increased contacts and queries and administrative time for Council staff responding to and following up information in relation to incomplete documentation. In response to the identification of these issues questionnaire respondents were asked to rate their level of understanding with each of the five Council questions. Respondents indicated on a sevenpoint Likert scale (the score of 1 indicated excellent understanding and the score of 7 indicated very poor understanding). Responses are presented in Table 9. 6303_nurse_competency copy copy.indd 59

60 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Table 9 Rate you understanding of each of the following questions 1 2 3 4 5 6 7 Excellent understanding Very Poor Overall Respondent Group Mean Standard Deviation Have you completed a minimum of 450 hours of nursing practice in New Zealand within the past three years? n 1,051 57 18 12 2 6 11 % 91% 5% 2% 1% 0% 1% 1% 1.20.815 Have you undertaken the minimum of required professional development hours (i.e. 60 hours) within the past three years? n 1,029 83 22 9 6 2 6 % 89% 7% 2% 1% 1% 0% 1% 1.19.703 Do you meet the Council's competencies for your scope of practice? n 941 122 59 20 6 7 2 % 81% 11% 5% 2% 1% 1% 0% 1.32 8.12 Do you have a mental or physical condition that means you are unable to perform the functions required for the practice of nursing? n 826 92 31 14 9 8 17 % 71% 8% 3% 1% 1% 1% 15% 2.15 2.185 Have you been the subject of an investigation, disci plinary or criminal proceedings or a disciplinary order in New Zealand or any other country since you last applied for a practicing certificate? n 883 58 14 9 3 4 186 % 76% 5% 1% 1% 0% 0% 16% 2.09 2.214 *Total respondents N = 1157, 95% Confidence Level, Margin of Error 2.85% The majority of respondents indicated an excellent understanding of the five questions (M = 1.19 2.09). However, it is of note that for two questions, Do you have a mental or physical condition that means you are unable to perform the functions required for the practice of nursing? and Have you been the subject of an investigation, disciplinary or criminal proceedings or a disciplinary order in New Zealand or any other country since you last applied for a practising certificate?, 18% (n = 208) and 17% (n = 202) of respondents respectively indicated they had a limited to very poor understanding of the meaning/intent of the questions. 6.4 Recertification audit In phases one and two of the research issues concerning of the role and function of the peer assessor was raised. To capture those issues the following questions were included in the questionnaire. In response to the question, Have you ever been asked to be a Peer Assessor for a colleague who was being audited? (Table 10), 21.80% (n = 252) responded yes, and 78.20% (n = 905) responded no. Of the respondents who indicated they had been a Peer Assessor, five identified as Enrolled Nurse and 247 as Registered Nurse. No Nurse Assistants were represented in the Peer Assessor group. 6303_nurse_competency copy copy.indd 60

NURSING COUNCIL OF NEW ZEALAND 61 Table 10 Have you ever been asked to be a Peer Assessor for a colleague who was being audited? Total Scope of Practice Nurse Assistant Enrolled Nurse Registered Nurse 1,157 7 45 1,105 Yes 252 0 5 247 21.80% 0.00% 11.10% 22.40% No 905 7 40 858 78.20% 100.00% 88.90% 77.60% Respondents were asked six questions in relation to being a Peer Assessor. Twenty-five per cent of respondents (n = 67) indicated they were not provided with information about the recertification audit process, and 18% (n = 48) indicated they were not provided with documentation about the relevant scope of practice and competencies. Ten per cent (n = 27) were not provided with a competence assessment form. When asked if the assessment was based on evidence, 6% (n = 15) indicated it was not. Fourteen per cent (n = 37) did not discuss their assessment with their colleague. Detailed results are depicted in Figure 11. Figure 11 Peer Assessor Of the overall respondent group 90% (n = 1,037) indicated they had not been selected for recertification audit. Ten per cent of the overall respondent group (n = 120) indicated they had been audited between 2005 and 2009 inclusive. Figure 12 depicts the audit distribution of respondents by audit year. Of the 120 respondents who indicated they had been audited, 111 identified as Registered Nurse and nine identified as Enrolled Nurse. No Nurse Assistants had been audited. 6303_nurse_competency copy copy.indd 61

62 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Figure 12 Recertification audit distribution of respondents by audit year Respondents were asked to respond to six questions with regard to the written information they received from the Council before their recertification audit. Table 11 When you were audited did you receive written information about? n = count of respondents selecting the option % is percent of the total respondents electing the option Total Yes No 1. The recertification audit process? 2. The recertification audit time frame? 3. The domains of practice and competencies for your scope of practice? 124 4. The evidence you would need to provide for the recertification audit? 125 5. Where you could obtain clarification if necessary? 6. The process after submission of your documentation? 127 125 124 123 n 116 11 % 91% 9% n 116 9 % 93% 7% n 114 10 % 92% 8% n 113 12 % 90% 10% n 102 22 % 82% 18% n 96 27 % 78% 22% 6303_nurse_competency copy copy.indd 62

NURSING COUNCIL OF NEW ZEALAND 63 Table 12 presents a cross-tabulation of responses by audit year. Responses to question 3, When you were audited did you receive written information about: The domains of practice and competencies for your scope of practice? improved significantly with 100% of respondents indicating they received the information in 2009. There was no significant variation in the responses received with regard to the other five questions across the five-year period. Table 12 When you were audited did you receive written information about 2005 2006 2007 2008 2009 1. The recertification audit process? Yes 100% 95% 100% 93% 93% No 0% 5% 0% 7% 7% 2. The recertification audit time frame? Yes 100% 100% 97% 97% 93% No 0% 0% 3% 3% 7% 3. The domains of practice and competencies for your scope of practice? 4. The evidence you would need to provide for the recertification audit? Yes 100% 90% 86% 90% 100% No 0% 11% 14% 10% 0% Yes 100% 90% 93% 90% 93% No 0% 11% 7% 10% 7% 5. Where you could obtain clarification if necessary? Yes 73% 90% 79% 83% 90% No 27% 11% 21% 17% 10% 6. The process after submission of your documentation? Yes 70% 84% 72% 87% 79% No 30% 16% 28% 13% 21% *Total respondents N = 120 Based on the documentation they had received from the Council, respondents were asked to rate their understanding of how to provide evidence for four items: practice hours; professional development hours; selfassessment of competencies for their scope of practice and peer assessment of competencies for their scope of practice. Responses to the four items are collated in Table 13. 6303_nurse_competency copy copy.indd 63

64 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Table 13 Based on the documentation provided to you by the Council, rate your understanding of how to provide evidence for each of the following requirements 1 2 3 4 5 6 7 Excellent understanding Very poor understanding Evidence of practice hours. n 129 16 10 5 5 0 0 Evidence of professional development hours. Self assessment of your competencies for your scope of practice. Peer assessment of your competencies for your scope of practice. % 78% 10% 6% 3% 3% 0% 0% n 119 22 10 5 5 0 3 % 73% 13% 6% 3% 3% 0% 2% n 86 30 16 13 7 6 6 % 52% 18% 10% 8% 4% 4% 4% n 84 25 17 14 10 7 6 % 52% 15% 10% 9% 6% 4% 4% Mean Standard Deviation 1.43.958 1.58 1.213 2.19 1.675 2.30 1.737 Respondents indicated on a seven-point Likert scale (the score of 1 indicated they had an excellent understanding and the score of 7 indicated they had a very poor understanding). The item Evidence of practice hours achieved a mean score of (M = 1.43) with a standard deviation of (SD =.958) indicating the overall respondent group had an excellent understanding of the documentation relating to how to provide evidence of practice hours. The other three items all achieved mean scores ranging from 1.58 to 2.30, indicating good to excellent understanding of the documentation by the majority of respondents. However, as depicted, the standard deviation ranged between 1.213 and 1.737, reflecting a greater distribution of scores across the seven-point rating scale. Based on a list of three options (A., B. & C.) respondents were asked to select which option best reflected the communication or documentation they received from the Council following submission of their audit material. The findings are presented in Table 14. Table 14 Following submission of audit documentation A. No further correspondence 93 66% B. Single correspondence requesting further information 32 23% C. Multiple correspondence 16 11% Total 141 100% Of the overall respondent group of 141, 66% (n = 93) indicated they received no further correspondence. Twenty-three per cent (n = 32) received a single correspondence requesting further information and 11% (n = 16) indicated they received multiple correspondence. Respondents were asked to rate their level of agreement with four satisfaction statements in relation to the recertification audit documentation, communication and process. Ratings were scored on a seven-point Likert scale (the score of 1 indicated they strongly agreed and the score of 7 indicated they strongly disagreed). The findings are presented Table 15. 6303_nurse_competency copy copy.indd 64

NURSING COUNCIL OF NEW ZEALAND 65 Table 15 Satisfaction with recertification audit documentation, communication and process 1 2 3 4 5 6 7 Strongly agree Strongly disagree Mean Standard Deviation The specified time frames were acceptable. The request for information and correspondence from the Nursing Council was clear. The style of correspondence from the Nursing Council was appropriate. n 60 29 20 11 5 3 9 % 44% 21% 15% 8% 4% 2% 7% n 62 29 19 12 5 5 6 % 45% 21% 14% 9% 4% 4% 4% n 62 24 17 17 6 8 2 % 46% 18% 12% 12% 4% 6% 1% 2.39 1.759 2.33 1.684 2.36 1.636 I was satisfied with the process. n 57 22 18 13 11 2 14 % 42% 16% 13% 9% 8% 1% 10% 2.72 1.989 Generally there was a high level of agreement with the four statements producing a range of means scores from (M = 2.33 2.72). However, there was a distribution of scores across the seven-point rating scale (SD = 1.636 1.989), with a small proportion of respondents indicating they strongly disagreed with each statement. 6.5 Professional Development and Recognition Programmes Three questions were included in the questionnaire with regard to Professional Development and Recognition Programmes. Whilst PDRPs are not the focus of this research, there was significant comment raised throughout the interview process (phase two) with regard to perceived advantages and disadvantages with regard to PDRP and the perceived overlap/link with the CCF. On several occasions the comment was made that PDRPs should be compulsory for all nurses. However, this was not a consistently held opinion. Respondents were asked if they believed PDRPs should be compulsory. respondents participated in this question. Eleven hundred and thirty-six Figure 13 Should Professional Development and Recognition Programmes be Compulsory? 6303_nurse_competency copy copy.indd 65

66 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Figure 13 presents the overall group response. Forty-nine per cent (n = 557) responded yes PDRPs should be compulsory and 51% (n = 579) responded no they should not. This response is representative of the nursing population with a margin of error of 2.87% and 95% confidence level. Table 16 presents a cross-tabulation of responses by employment setting. It is interesting to note that of the respondents employed by DHBs an employment setting in which respondents generally have access to PDRP (n = 309) responded yes they should be compulsory and (n = 330) responded no they should not Table 16 Cross-tabulation of Employment setting by Should PDRP s be compulsory? DHB (Acute) DHB (Primary Health/Community) DHB (Other) Private Hospital Primary health (NGO / PHO) PHO Aged Care Sector (Rest home / Residential Care) Nursing Agency Self Employed Maori Health Service Provider Rural Health Management Educational Institution Government Agency (MOH, ACC, Corrections Service, Defence Forces) Other please specify Yes No 1,124 458 97 84 84 78 21 96 14 13 9 7 15 29 15 104 553 228 43 38 38 36 10 56 8 5 5 3 9 11 11 52 49% 50% 44% 45% 45% 46% 48% 58% 57% 39% 56% 43% 60% 38% 73% 50% 571 230 54 46 46 42 11 40 6 8 4 4 6 18 4 52 51% 50% 56% 55% 55% 54% 52% 42% 43% 62% 44% 57% 40% 62% 27% 50% *Total respondents N = 1124, 95% Confidence Level, Margin of Error 2.89% Respondents were asked to indicate if they were levelled on a PDRP and if they had access to a PDRP. Eleven hundred and seventeen respondents participated in this question. Fifty-six per cent (n = 626) indicated they were levelled on a PDRP and 44% (n= 491) indicated they were not. Figure 14 displays the results. 6303_nurse_competency copy copy.indd 66

NURSING COUNCIL OF NEW ZEALAND 67 Figure 14 Levelled on a PDRP - Have access to a PDRP In response to the question Do you have access to a PDRP? 1,129 respondents participated. Seventy-six per cent (n = 863) responded that they had access to a PDRP and 24% (n = 266) responded that they did not have access. Twenty per cent of the respondents who have access to a PDRP have chosen not to be levelled. 6.6 Summary of findings from the questionnaire This section summarises and highlights the findings from the questionnaire. These will be discussed in more detail in chapter seven where the results are triangulated. 6.6.1 Demographic data Respondent sample is representative of overall population of nurses who hold current practising certificates, with a 2.85% margin of error and 95% confidence level. Respondents represent a diverse demographic sample in terms of identified employment setting and practice area. 6.6.2 Competence and fitness to practise The majority of respondents (76%) believe the Council s Continuing Competence Framework and processes for renewing practising certificates provide the mechanism to ensure nurses are competent and fit to practise. A combination of the three continuing competence indicators (self-declaration, practice hours and continuing professional development) was ranked as the best evidence of competence to practise and ongoing professional development. The self-declaration, if used independently, was ranked as the worst indicator of competence. Seventy per cent of respondents strongly agree that individual nurses are responsible for maintaining their own competence to practise. Sixty-one per cent believe their employer is responsible for maintaining their competence to practise. Forty-one per cent believe the Council is responsible for maintaining their competence to practise. 6303_nurse_competency copy copy.indd 67

68 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Ninety-eight per cent of nurses indicated they understood that the self-declaration is a legal document. The majority of nurses indicated they understood the statements used on the Council application for renewal of practising certificates, with the exception of Do you have a mental or physical condition that means you are unable to perform the functions required for the practice of nursing? and Have you been the subject of an investigation, disciplinary or criminal proceedings or a disciplinary order in New Zealand or any other country since you last applied for a practising certificate? Eighteen per cent and 17% respectively indicated they had a poor understanding of the meaning of these questions. 6.6.3 Recertification audit Twenty-one per cent of the overall respondent group indicated they had been a peer assessor. Of the peer assessor group 25% indicated they were not provided with information about the process, 18% were not provided with documentation about the scope of practice, 10% were not provided with assessment forms, 6% indicated their assessment was not based on evidence and 14% did not discuss the assessment with the colleague they assessed. Ten per cent of the overall respondent group had been audited between 2005-2009 inclusive. The majority of audit participants received written information with regard to the audit process, time frames, competencies, evidence, where to obtain clarification and the process after submission of documentation. A small percentage indicated they did not. The majority of respondents indicated they had a good understanding of what evidence to provide for the recertification audit based on the information they received from the Council. Thirty-four per cent of respondents indicated they received requests for further information following submission of their audit documentation. Participants indicated they were generally satisfied with the recertification audit documentation, communication and process. Forty-nine per cent of respondents indicated PDRPs should be compulsory and 51% indicated they should not. Fifty-six per cent of respondent were levelled on a PDRP. Seventy-six per cent of respondents indicated they have access to a PDRP. 6.7 Conclusion This chapter has presented the questionnaire findings. A number of the findings provided confirmation of the views expressed by interview respondents. Chapter seven will discuss and triangulate the findings from each previous phase of the research in relation to the research objectives. Finally recommendations will be made for ongoing development and quality improvement of the CCF. 6303_nurse_competency copy copy.indd 68

NURSING COUNCIL OF NEW ZEALAND 69 Chapter Seven Discussion, Conclusion and Recommendations 7.1 Introduction This chapter discusses and triangulates the research data and findings from the previous research phases and presents them in association with the research outcomes identified by the Council (NCNZ, 2008). The chapter will conclude by presenting a summary of key research findings and recommendations. 7.2 Data triangulation and discussion The overwhelming consensus of key stakeholder respondents was that the CCF is a critical and important mechanism to ensure nurses are fit and competent to practise. Seventy-six per cent of questionnaire respondents believe the Council s CCF and processes for renewing practising certificates provide the mechanism to ensure nurses are competent and fit to practise. This response is representative of the nursing population with a margin of error of 2.85% and confidence level of 95%. A number of items for clarification and or quality improvement have been highlighted in the data and will be further analysed and discussed in this chapter. The report touches briefly on the confusion that the interview data seem to indicate exists between the evidential requirements of the PDRP, the evidentiary requirements of the CCF recertification audit, the notion of mandatory versus voluntary participation in PDRP and role confusion between the responsibility of the Council, employers and the individual nurse. 7.2.1 Purpose, roles and responsibilities The international literature is unequivocal about the importance of CCFs (ANMC, 2007; Bryant, 2005; Chiarella, 2006; CNA, 2000; EdCaN, 2008; FitzGerald, et al., 2001). Importantly, CCFs demonstrate to the public that the regulatory authority and nursing profession are cognisant of and have mechanisms to assess the continued competence of the profession and ensure public safety. CCFs also promote consistency of continuing competence standards and assessment, and provide a mechanism for the assessment of competence as a measure of public safety. Further, framework standards and assessment options should be flexible, have relevance and be transferable to the differing levels of practice and settings in which nurses practise, and assessment should be mandatory for all members of the profession. Interview respondents clearly supported the Council s CCF as a mechanism of setting standards and the notion of capturing practice development. One respondent commented The continuing competence framework is a process whereby Nursing Council has set some standards for the amount of professional development that nurses have to undertake in order to maintain their APC. Also, the amount of time that they need to be in actual practice in order to maintain their APC. So it s a way of creating a standard which replaces the previous system of having no standard of what people were engaged in. However, respondents raised the issue of confusion regarding the purpose of the Continuing Competence Framework and roles and responsibilities inherent in the process, suggesting that, while the Council has a role in ensuring some degree of competence, there is a general misunderstanding amongst nurses regarding the Council s role. 7.2.2 The nature of a protective jurisdiction and the role of the regulatory authority As discussed in chapter one and identified by interview respondents, the role of the Council - the regulatory authority established to administer the legislation in relation to nurses in New Zealand - is often misunderstood by nurses and, indeed, other health professionals. The role of the legislation (HPCA Act 2003) is protective, and therefore the institutions, roles and committees created by it exist to protect the public from the risk of harm, rather than to protect the interests of the professions so regulated. The functions and powers of the Council are defined in the legislation and establish a form of regulatory regime known as a protective jurisdiction 6303_nurse_competency copy copy.indd 69

70 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK (Staunton & Chiarella, 2008, pp. 213-214). Hence, the role of the Council among other things is to establish and maintain standards of practice. This includes setting the standards for monitoring the competence and continued competence of the profession to ensure public safety. Hence compliance of individual nurses with the requirements of the CCF is mandatory. Any health practitioner who is concerned about another health practitioner s practice and who considers the standard of practice may pose a risk of harm to the public may notify the Registrar of the relevant authority (s34 HPCA Act 2003). There are also provisions for both the public and specified health providers to notify the Registrar about health concerns that may affect a health practitioner s ability to practise (s45 HPCA Act 2003). Clearly if such concerns are raised, even though regulated health professional groups enjoy a respected public profile, an important aspect of that regulation is that health professionals who are part of a regulated professional group can be brought to account for their practice and, if their registration is cancelled, lose their right to practise. In New Zealand, nurses are held accountable for their practice under the Health Practitioners Competence Assurance (HPCA) Act 2003 and are expected to maintain appropriate standards of professional conduct. Were these standards to be breached, then a complaint may be lodged against that nurse. However, having recognisable standards of practice that are upheld by the professional disciplinary bodies also provides guidance for the profession and assists in setting boundaries for professional practice. In a New South Wales (NSW) (Australia) Supreme Court decision Health Care Complaints Commission v Bruce Litchfield, (1997, 41 NSWLR 630) the Court explained that: Disciplinary proceedings against members of a profession are intended to maintain proper ethical and professional standards, primarily for the protection of the public, but also for the protection of the profession (ibid. 635). It is important to understand where this area of law sits within the legal system, as this protective jurisdiction is an area that is often misunderstood and can cause confusion. This is particularly the case when other areas of the legal system are also involved in the matter, such as the criminal jurisdiction, coronial jurisdiction or the civil law area of negligence. A protective jurisdiction forms part of a body of law known as administrative law, a branch of law which deals with the administrative processes of governments and formal decision-making bodies. It has very different functions and processes from the criminal law which exists to punish offenders and to deter potential offenders (Bates, 1989). The NSW Supreme Court in HCCC v Litchfield went on to say it accepted that the toll of disciplinary proceedings might be high in terms of money and emotional stress, but took pains to explain that this was not the intention of a protective jurisdiction. These matters would be highly relevant if the purpose of these proceedings were punitive, but their purpose is entirely protective. In Clyne v NSW Bar Association (1960) 104 CLR 186 at 201-2 the Court said: Although it is sometimes referred to as the penalty of disbarment it must be emphasised that a disbarring order is in no sense punitive in character. When such an order is made, it is made, from the public point of view, for the protection of those who require protection, and from the professional point of view, in order that abuse of privilege may not lead to loss of privilege (p. 635). In Condon (NMT230206JHC ) a NSW Nurses Tribunal case, the Tribunal explained the nature of a protective jurisdiction as follows: The major issues which concern this Tribunal in terms of protective orders are concerned with maintaining the public standing of and public confidence in the nursing profession, maintaining standards within the profession and that of general deterrence to make it clear that the type of behaviour in which [the nurse] engaged,... is not acceptable behaviour for a registered nurse (p.13). It is hoped this discussion assists in developing an understanding of the nature of a protective jurisdiction. The aim is not to punish the nurse, although it might be argued that suspension or de-registration certainly does. However, this is a by-product of the legislation in a protective jurisdiction, not its primary intent. This is an important issue for future discussion of this report, as it is apparent that there is significant misunderstanding 6303_nurse_competency copy copy.indd 70

NURSING COUNCIL OF NEW ZEALAND 71 about the role of the Council, with several comments suggesting that it exercises a punitive jurisdiction over the nursing profession, rather than a protective jurisdiction for the public. 7.2.3 What is meant by the requirement for continuing competence? Within any discussion about the requirement for continuing competence it is important to differentiate between the original requirement for competence on initial registration and the Continuing Competence Framework. It is clear from our data (see chapter five) that there is a lack of clarity about what the required standard of continued competence is. Several respondents indicated that the Council should only be concerned with the minimum standard of competence and that the employers role is to drive a performance development culture. Some of the interviewees insisted the standard to be met was the minimum standard and one even suggested they only had to meet the same standard as they met on registration. This does not correspond to the requirement of reasonableness set out in s5(1) of the HPCA Act 2003. The requirement is about the need to demonstrate that you continue to be competent to a standard reasonably expected in your scope of practice. How you live out this scope of practice would have to be dependent upon the roles in which you work otherwise no one could ever be held to account, nor would there be any need to differentiate between nurses working in management, research, education, policy or the like. It is not enough for a Director of Nursing to demonstrate he/she has met the competencies on review in 2010 in the same way as he/she met them on graduation in 1990. His/her competence today is about the ability to fulfil a management role. Similarly, if someone were a manager and a clinician, or a manager and an educator, the evidence they would submit to demonstrate continuing competence would not be that they attended CPR training and an IV accreditation, although arguably both of those would be valuable for the new graduate working in the emergency department. But the requirement for the manager/educator/researcher would be relevant work experience for them and relevant CPD for them. In a protective jurisdiction, that which is reasonable must always correlate with that which keeps the public safe. General discussion with respondents about assessment of continuing competence demonstrated a lack of consensus about what should be assessed and the level of assessment required. As previously discussed, health professionals are expected to maintain appropriate standards of professional conduct. This does not equate with the notion of a minimum standard. There is also some confusion identified with regard to who is responsible for maintaining competence to practise. In response to the web-based questionnaire, 70% of respondents strongly agreed that individual nurses are responsible for maintaining their own competence to practise. However, 61% also believed their employer is responsible for maintaining their competence to practise, and a further 41% believed the Council is responsible for maintaining their competence to practise. Cross-tabulation of the data by employment area indicated that nurses employed in Health Management (n = 15) and Educational Institutions (n = 29) scored the highest mean scores for the item my employer is responsible for maintaining my competence to practise and item NCNZ is responsible for maintaining my competence to practise. Whilst this response does not appear to be unique to the New Zealand context (ANMC, 2007; Campbell & MacKay, 2001; Chiarella, et al., 2008; CNA, 2000; FitzGerald, et al., 2001; Goodridge, 2007), it does highlight the need for the Council to be explicit and overt in terms of the purpose of the CCF, the role and responsibility of the Council, and the responsibility of the individual Health Professional (Nurse), the employer and the profession in this regard. In phases one and two of the research, misinterpretation of the intent/meaning of the self- declaration questions listed on the Council s Application for Practising Certificate form was introduced as a possible issue in terms of nurses not completing the documentation accurately. It was felt this aspect called into question the veracity and validity of the self-declaration and as a result would increase contact (queries and administration) with Council staff and administrative time spent following up information in relation to incomplete and inaccurate documentation. Ninety-eight per cent of respondents indicated they understood that the self-declaration is a legal document and yet in response to the question relating to indicators of competence, self- declaration was scored as the worst indicator of competence. Arguably if a health professional completes an honest and subjective selfassessment, as required when completing the current Application for Practising Certificate form, then signs the declaration indicating their competence and fitness to practise, it should be a valid and verifiable indication 6303_nurse_competency copy copy.indd 71

72 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK of their competence and hence safety to practise. However, a number of interview respondents questioned the validity of the declaration made by nurses when completing the Application for Practising Certificate. They suggested that its status and significance is not apparent to some nurses and the form is seen purely as a tick box unless the nurse is actually selected for recertification audit and required to provide validated evidence. 7.2.4 Communication and consultation In terms of communication and consultation, the literature identifies that clear articulation of the purpose of the CCF is required (public protection or public protection and lifelong learning). There must also be clear articulation of continued competence standards, documentation related to the CCF must be accessible and processes transparent, and web-based options should be available (ANMC, 2007; Chiarella, 2006; Chiarella, et al., 2008; CNA, 2000; Goodridge, 2007; McGrath, et al., 2006). Involvement of all levels of the profession is also noted as essential for CCFs (ANMC, 2007; CNA, 2000). It is clear there has been considerable consultation with the profession and other key stakeholders over a number of years that various iterations of the CCF were developed and implemented. Nurses have been kept well informed. The Council s requirements for renewal of practising certificates were provided individually to all nurses in the form of a Council News Update, dated 1 November 2004. This newsletter contained a summary of the HPCA Act 2003, the Council definition of practising, the scopes of practice, process for annual practising certificate renewal, the competence and fitness to practise requirements for renewal of practising certificates, and an overview of the CCF. The following section presents a discussion of the findings associated with the research outcomes stipulated by the Council. 7.3 Research outcomes and discussion 7.3.1 Explore the validity of the stipulated hours of professional development and day/hours of clinical practice over a three- year period as indicators of competence As identified in the literature (ANMC, 2007; Bryant, 2005; CNA, 2000) frameworks are tools that have a role in regulating and guiding the profession by setting the standards for competence assessment and ensuring consistency in the monitoring and ongoing assessment of competence (Pearson, Fitzgerald, Walsh, et al., 2002). They have a clear purpose in terms of public protection. However, literature suggests that if their purpose is also to promote lifelong learning then this must be clearly articulated (ANMC, 2007; Campbell & MacKay, 2001; Goodridge, 2007) as it will influence the level of assessment required. The international literature identifies that the most commonly used indicators of competence are selfassessment, peer assessment, recency of practice and continuing professional development/education. A combination of indicators is recommended no one indicator used independently can measure competence. Valid measurement of indicators which are subjective in nature is difficult. Inter-rater reliability is a critical component of the assessment process. Recency of practice and practice tasks are quantifiable indicators, but should not be used in isolation (ANMC, 2007; Campbell & MacKay, 2001; FitzGerald, et al., 2001). It is also of note that despite clear and reasonably consistent definitions of competence articulated by a number of nurse regulatory authorities, a level of confusion about the conceptualisation of competence and the distinction between core and higher levels of competence, behaviours and insight is still apparent (ANMC, 2009; Campbell & MacKay, 2001; CNA, 2000; Goodridge, 2007; Nursing and Midwifery Council, 2008). Following a number of consultative processes, the Council decided in August 2004 to implement the indicators of competence. The CCF includes three indicators of competence: A. Self-declaration of competence to practise (based on self-appraisal using the Council competencies for the relevant scope of practice); B. Verification of practice hours (minimum of 450 hours/60 days in the past three years); C. Verification of professional development (minimum of 60 hours in the past three years). Qualitative data revealed there was some criticism of the notion of hours as a reflection of competence. Issues identified included the actual number of hours in comparison with other health professions, the determination 6303_nurse_competency copy copy.indd 72

NURSING COUNCIL OF NEW ZEALAND 73 of the hours and elements that might be captured in a portfolio to indicate competence. The idea that indicators might actually represent competence was much less clear, with some comment about validity and reliability. There was also a range of comments about what constituted professional development and the role and accountabilities of peer assessors. In the survey, respondents were asked to rank the indicators from 1 (Best) to 7 (Worst) to indicate which they believed provided the best evidence of continuing competence to practise. The indicator of competence to practise ranked best by respondents (n = 470) was the combination of the self-declaration (A), evidence of practice hours (B) and evidence of ongoing professional development (C), which represented 52% of the total overall responses. The least popular indicator of competence to practise was (A) self-declaration only, which represented a response from (n = 311) 40% of the total respondent group. Respondents ranked a combination of the three indicators (A, B & C) as providing the best evidence of competence to practise and ongoing professional development. However, it is of note that the respondents ranked the self-declaration of competence (based on self-appraisal of their competence using the Council competencies for their scope of practice) as the worst indicator for providing evidence of competence to practise when used independently. Data from interviews and the survey indicate there is a general satisfaction with the stipulated hours for professional development and for clinical practice. However, it is clear that, in terms of competence, these stipulated hours are indicators rather than guarantees. 7.3.2 Provide information on the efficacy of undertaking a random audit of five per cent of the nursing workforce to meet recertification requirements In December 2005, as part of the CCF framework, the Council initiated the recertification audit process of individual nurses. Five per cent of nurses renewing their practising certificates annually are randomly selected for individual recertification audit. The decision by the Council to select five per cent of nurses for audit annually appears to have been based on available international literature and best practice at the time. This figure is pragmatic, as no single piece of evidence exists to prove it is a valid representation of the New Zealand nursing workforce; however, statistical findings from the recertification audits and competence notifications conducted over the past five years suggest the five per cent measure is appropriate and effective. As a quality feedback loop, it is useful as a validation tool. Five interview respondents had been audited. Concerns were raised about the transparency of the audit process, for example who would be assessing the material provided. The respondent comment below reflects the discussion There was no indication of how it would be assessed or who would assess it...[it] really concerned me that I had no idea of who was going to be viewing this information... So I did worry about the notion of confidentiality. Another respondent was concerned that evidence submitted for recertification was personally identifiable, and commented It wasn t an objective - well it wasn t an assessment that was any nurse making application, it was because I was identifiable. The comments were specific because they knew who I - what my role was. There was also concern expressed about the lack of feedback about the audit process. Inter-rater reliability and moderation processes in relation to the assessment of evidence are not apparent in the documents reviewed. The need for processes that ensure inter-rater reliability is supported by the international literature (McGrath, et al., 2006). Data from the web-based questionnaire indicated that 10% of the overall respondent group had been audited between 2005-2009 inclusive. The majority of audit participants indicated they received written information with regard to the audit process, timeframes, competencies, evidence, where to obtain clarification and the process after submission of documentation. A small percentage indicated they did not. Whilst the percentage 6303_nurse_competency copy copy.indd 73

74 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK is small in terms of the overall nursing population, it should be noted that it still represents a significant cost in terms of the Council s administrative resources responding to and processing additional queries, requests and documentation. The majority of respondents indicated they had a good understanding of what evidence to provide for the recertification audit based on the information they received from the Council. Thirty-four per cent of respondents indicated they received requests for further information following submission of their audit documentation. This is another area that represents a significant cost in terms of the Council s administrative resources. Overall respondents indicated they were generally satisfied with the recertification audit documentation, communication and process. Additionally, in response to the question when you were audited did you receive written information about the domains of practice and competencies for your scope of practice?, there was significant improvement in 2009 from previous years with 100% of respondents indicating they received the information. Nurses who participate in a Council-approved PDRP process are exempt from the recertification audit process. Council documentation relating to the implementation of the CCF and recertification audit process signals that recertification audit numbers were expected to drop significantly over the initial five-year period, because of increased participation of nurses in PDRPs. Whilst there has been some reduction (16.5%) in recertification audit numbers (n = 1,288, 2006-2007; n = 1,075, 2008-2009) this has not been as much as anticipated and may be due to voluntary participation in PDRPs and relatively low participation nationally. 7.3.3 Document and track the different forms of written evidence that is currently acceptable to the Council to demonstrate competence The Council has three policy documents associated with the CCF: these are the Continuing Competency (GPO02.10) (August 2004, March 2007), the Continuing Competence (RP05.03) (August 2004, June 2009), and the Recertification audit process policy (GPO 05.03) (August 2006, May 2008). These are in-house documents. The Continuing Competency (GPO02.10) is a governance document and the Continuing Competence (RP05.03) and the Recertification audit process policy (GPO 05.03) outline the procedural requirements of the CCF and recertification audit process. Both are written in a guideline/procedural format rather than that of a formal policy, and are not publicly available. Whilst they include a policy statement they do not include a purpose statement and focus solely on procedural aspects of the CCF and recertification audit processes. However, a clear explanation of criteria for the CCF, exemption from the recertification audit, and the recertification audit evidential requirements is available to the public on the Council s website. Again there is no purpose statement linking the CCF to the Council requirements mandated by the HPCA Act 2003. In addition there is no information available to nurses with regard to the recertification process or timelines once their documentation has been submitted to the Council for assessment. This is an area of concern which was raised by interview respondents and confirmed by questionnaire respondents. As noted earlier a number of iterations of different forms of written evidence to demonstrate competence were developed by the Council. One early option was for nurses to maintain a personal professional portfolio of evidence which could be called on for audit. It appears the later decision not to audit portfolios was pragmatic and largely based on logistical and economic considerations. This decision is supported by international literature that suggests portfolio is not an adequate indicator of continuing competence or safety to practise due to its subjective nature and lack of inter-rater reliability. It is a tool best used to record practice and develop an individual s reflective thinking/practice. (ANMC, 2007; CNA, 2000; EdCaN, 2008; FitzGerald, et al., 2001). Currently a package of hard copy documentation is posted to individual nurses selected for recertification audit. This package includes: a form letter recertification audit information sheet why have I been selected for a recertification audit? nurse audit checklist. 6303_nurse_competency copy copy.indd 74

NURSING COUNCIL OF NEW ZEALAND 75 Nurses are directed to the Council website to download and print the relevant template documents, for example competencies for the scope of practice and competence assessment forms. These documents are not included as hard copy in the recertification audit package and are only available as a PDF template from the website. Hence nurses are required to source the relevant documentation and enter their evidential data by hand rather than being provided with the opportunity to download and enter their data electronically or reformat as a word document so that information can be typed in. The notion that the Council s current process for competence is just an indicator of competence was expressed by one respondent... an audit done by a regulatory body in no way can say... Oh you re competent to practise. A PDRP process I think can because it goes into much more depth. But anything a regulatory authority does can only be an indicator really. Additionally, there is confusion with the Council s role in recertification and Professional Development and Recognition Programmes (PDRPs) which are approved by the Council, which was highlighted in the following quote Nursing Council should actually clarify the purpose of the process Because it has, it s got confused with PDRP. Without a shadow of a doubt. The idea of it being minimal and they ve - and it s a - just an absolute focus on safety. Public safety as opposed to professional development. As demonstrated the requirements for continuing competence, the Council s CCF and the PDRP have become enmeshed and subsequently have created confusion. 7.3.4 Identify issues related to peer assessment of competence The decision to include peer assessment as part of validation of competence was based on information provided by the United Kingdom Nursing and Midwifery Council (UKCC) in its publication titled the PREP Handbook. The UKCC used peer assessment as a means of validation for self-assessment. In relation to peer assessment for validation of comments, a range of issues were reported in the qualitative data. The majority of responses indicated the peer assessors should be a nurse. However, there was also comment that, in some situations, it may be appropriate that another health practitioner who works closely with a nurse could undertake peer assessment but with the proviso that there needed to be clear and explicit assessment criteria....well it s hard to know if they re valid because people will interpret what evidence is required for them the best way they can. I think there could be more guidelines. Peer assessments I think that - that in itself lends it a whole new perspective on people who might sign off that somebody else is competent because they are - they need the staff. And they also - I mean I suppose I could say I know of some cases where people have signed off people as being competent with a peer review, or validated stuff when it s been thrust under their nose. Concerns were also evident in the findings from questionnaires. In response to a question about whether respondents had ever been asked to be a peer assessor, 21% of the overall respondent group indicated they had been a peer assessor. A further question sought information about a number of issues identified in the interviews: provision of information; provision of documentation about the relevant scope of practice and competencies; provision of a competence assessment for a colleague; if the assessment was based on evidence; if the assessment was discussed with the colleague and understanding that what was being completed and signed was a legal document. Of the peer assessor group, 25% indicated they were not provided with information about process, 18% were not provided with documentation about the scope of practice, 10% were not provided with assessment forms, 6% indicated their assessment was not based on evidence and 14% did not discuss the assessment with the colleague they assessed. 6303_nurse_competency copy copy.indd 75

76 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 7.3.5 Develop a framework to enable the Council to complete a further evaluation in five years time Despite the limitations of this evaluation research there is potential to replicate this model of mixed-methods evaluation for future evaluation of the CCF in five years time. Sufficient methodological description has been added to the report to make transparent the design and process. This should enable replication of all three phases of this research, the literature and document review, the interviews and electronic survey. As the first two stages of the project informed the survey design it is possible that a future replication of this study would be concerned with different survey questions or emphasis. It should be acknowledged that the CCF and, indeed, evaluation research are evolving and may both have progressed significantly in five years time. In addition future researchers may wish to benchmark with similar CCF international models. It would be possible to undertake some comparative analysis of this base data as more regulatory authorities start to develop their frameworks for evaluation of continuing competence standards. 7.4 Key findings The Continuing Competence Framework (CCF) The overwhelming consensus of respondents was that the CCF is a critical and important mechanism to ensure nurses are fit and competent to practise. Seventy-six per cent of survey respondents believe the Council s CCF and processes for renewing practising certificates provide the mechanism to ensure nurses are competent and fit to practise. There is historical evidence that the development of the CCF was well researched and included extensive consultation with stakeholders. Research outcomes 7.4.1 Explore the validity of the stipulated hours of professional development and days/hours of practice over a three-year period, as indicators of competence. Following a number of consultative processes, the Council made a decision in August 2004 to implement three indicators of competence: 1. Practice hours (minimum of 450 hours/60 days in past three years); 2. Professional development (minimum of 60 hours in the past three years); 3. Assessment of competence to practise against the Council competencies for the relevant scope of practice. Data from interviews and the survey indicate there is general satisfaction with the stipulated hours for professional development and practice. Nurses were asked to rank the indicators which they believed provided the best evidence of continuing competence to practise. The indicator of competence to practise ranked best by 52% of respondents (n = 470) was the combination of the self-declaration, evidence of practice hours and evidence of ongoing professional development. 7.4.2 Provide information on the efficacy of undertaking a random audit of five per cent of the nursing workforce to meet recertification requirements. Five per cent of nurses renewing their practising certificates annually are randomly selected for individual recertification audit. This figure is pragmatic, as no single piece of evidence exists to prove it is a valid representation of the New Zealand nursing workforce. However, statistical findings from the recertification audits conducted over the past five years suggest the five percent measure is appropriate and effective. Recertification audit is generally considered by respondents to be an important quality indicator that provides a measure of validity and reliability to the CCF. 6303_nurse_competency copy copy.indd 76

NURSING COUNCIL OF NEW ZEALAND 77 7.4.3 Document and track the different forms of written evidence that are currently acceptable to the Council to demonstrate competence. The current documentation provided to nurses selected for recertification audit was examined. The interviews identified a number of quality improvement issues with regard to access to recertification audit documentation, guidelines and submission and assessment of audit materials. Issues were identified with regard to the validity and reliability of the self-declaration, self-assessment, peer assessment and assessment of Council evidence for the recertification audits. Recommendations That the CCF documentation available to nurses is revised to provide more explicit and detailed guidelines with regard to CCF evidential requirements, assessment processes, recertification audit process including timeframes for the recertification assessment and notification of the outcomes of the audit. Explore the potential to provide recertification audit material and guidelines in both hard copy and electronic formats, with the provision for participants to enter data directly onto electronic forms. Investigate a system of electronic submission recertification audit data. Develop a system of electronic tracking of recertification audit documents, accessible to nurses who are participants in the audit process. Instigate clear internal moderation processes to improve inter-rater reliability and transparency of audit processes. 7.4.4 Identify issues related to peer assessment of competence The nurses selected for the recertification audit are required to complete two of the following forms of assessment: self, peer or senior nurse. There is a lack of clarity and some confusion in regard to the required standard for assessment of competence. In response to a question about whether survey respondents had ever been asked to be a peer assessor, 21% of the overall respondent group indicated they had been a peer assessor. Of the peer assessor group, 25% indicated they were not provided with information about process, 18% were not provided with documentation about the scope of practice, 10% were not provided with assessment forms, 6% indicated their assessment was not based on evidence and 14% did not discuss the assessment with the colleague they assessed. Recommendations Provide on all CCF-related documentation (including the Council website) a clear and consistent definition of what constitutes continuing competence with explicit criteria of how continuing competence may be assessed. Revise the criteria and guidelines for the selection of peer assessors. Provide peer assessors with guidelines for the peer assessment process in addition to clear and explicit assessment criteria. Provide documentation options in hard copy or electronic format. 7.4.5 Other findings It was reported by some interviewees that there was a lack of clarity with regard to the role, responsibility and accountability of the Council, the individual nurse and the employer in terms of ensuring continuing competence to practise and public safety. It was reported that some interviewees indicated there was confusion between the evidential requirements of the CCF recertification audit, PDRP, the evidential requirements of the PDRP and the role of the Council, employers, and the individual nurse in this regard. 6303_nurse_competency copy copy.indd 77

78 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Questions were raised with regard to the verification and legal status of the self-declaration on the application for Annual Practising Certificate form. Recommendations Revise the CCF and Recertification Audit policy documentation to include a clear purpose statement and policy framework principles. Improve and make overt the public nursing profile of the Council with regard to its role and responsibility as the regulatory authority for nurses in New Zealand. Differentiate and communicate the Council s expectations with regard to the responsibility of individual nurses, employers and the profession in terms of the requirements of the CCF and the HPCA Act 2003. Clearly articulate and communicate the Council s role and responsibilities with regard to PDRP. Provide a clear and more comprehensive definition of the status of the self-declaration on the Application for Practising Certificate form. Reformat the Application for Practising Certificate form to make the crucial information with regard to the self-declaration more explicit and obvious. Clearly articulate the penalties for providing false and misleading information on all documentation related to the CCF. 7.5 Conclusion The purpose of this research was to undertake an evaluation of the Council s Continuing Competence Framework to meet the following outcomes: 1. explore the validity of the stipulated hours of professional development and days/hours of practice over a three-year period, as indicators of competence. 2. provide information on the efficacy of undertaking a random audit of 5% of the nursing workforce to meet recertification requirements. 3. document and track the different forms of written evidence that are currently acceptable to the Council to demonstrate competence. 4. identify issues related to peer assessment of competence. 5. develop a framework to enable the Council to complete a further evaluation in five years time. A sequential mixed-methods evaluation was used to undertake the research and was completed in four phases over an eight- month timeframe. Each sequential phase focused on a particular evaluand grouping, which served to inform the basis for the next phase of data collection and analysis. The findings of this research demonstrate that the CCF is an important, valuable and relevant process. Additionally, the CCF is considered to have appropriate indicators for continuing competence. There are no major changes suggested to the three elements of the CCF the self-declaration; the requirement for a minimum number of hours of continuing professional development and the requirement for a minimum number of hours of recent practice in the applicant s relevant scope of practice. However, it is important to note that the indicators in the CCF cannot guarantee that a nurse is safe to practise on any given day. Although there is overarching endorsement for the CCF, a number of areas have been identified where there is confusion about processes, roles and responsibilities. This report has made suggestions for improvement and also drawn upon the international literature to provide a number of recommendations to the Council to address these areas for improvement. 6303_nurse_competency copy copy.indd 78

NURSING COUNCIL OF NEW ZEALAND 79 References ANMC (2007). Development of a national framework for the demonstration of continuing competence for nurses and midwives Literature Review. Canberra, Australia: Australian Nursing and Midwifery Council. ANMC (2009). Continuing Competence Framework. February 2009 (p. 12). http://www.anmc.org.au/professional_ standards: Australian Nursing and Midwifery Council. Bates, P. W. (1989). An Overview of Professional Liability in Health Care. Keynote Lecture presented by invitation to the Annual Seminar of the Medico-Legal Society of Singapore, Oct 14-15, 1989. Singapore. Benner, P. (1984). From novice to expert: Excellence and power in clinical nursing practice. London: Addison- Wesley. Bryant, R. (2005). Regulation, roles and competency development. The Global Nursing Review Initiative. Geneva: International Council of Nurses. Burgess, M. E. (2008). A guide to the law for nurses and midwives (4th ed.). Auckland: Pearson Education. Calman, L., Watson, R., Norman, I., Redfern, S., & Murrells, T. (2002). Assessing practice of student nurses: methods, preparation of assessment and student views. Journal of Advanced Nursing, 38(5), 516-523. Campbell, B., & MacKay, G. (2001). Continuing Competence: an Ontario nursing regulatory programme that supports nurses and employers. Nursing Administration Quarterly, 25(2), 22-30. Carnwell, R. (1997). Evaluative research methodology in nursing and healthcare. London: Churchill Livingstone. Casswell, S. (1999). Evaluation research. In C. Davidson & M. Tolich (Eds.), Social science research in New Zealand: many paths to understanding (pp. 198-207). Auckland: Pearson Education. Centre for Innovation in Professional Health Education and Research (2007). Review of work-based assessment methods. Sydney: The University of Sydney. Chappell, A. (1995). Competencies for practice. Paper presented at the National Nurses Forum Evidencebased Practice, Waipuna Conference Centre, Auckland, New Zealand. Chiarella, M. (2006). Competence in nursing practice: a controversial concept a focused review of literature. Sydney: NSW Health. Chiarella, M., Thoms, D., Lau, C., & McInnes, E. (2008). An overview of the competency movement in nursing and midwifery. Collegian, 15(2), 45-53. Clifford, C. (1997). Qualitative research methodology in nursing and healthcare. London: Churchill Livingstone. CNA (2000). A national framework for continuing competence programmes for registered nurses. September 2009. Ottawa: Canadian Nurses Association. Cowan, D. T., Norman, I. J., & Coopamah, V. P. (2005). Competence in nursing practice: a controversial concept: a focused review of literature. Nurse Education Today, 25(5), 355. Davidson, E. J. (2005). Evaluation methodology basics. Thousand Oaks: Sage Publications. Davis, R., Turner, E., Hicks, D., & Tipson, M. (2008). Developing competency framework for diabetes nursing. Journal of Clinical Nursing, 17(2), 168-174. Dolan, G. (2003). Assessing student nurse clinical competency: will we ever get it right? Journal of Clinical Nursing, 12, 132-141. 6303_nurse_competency copy copy.indd 79

80 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK EdCaN (2008). Competency Assessment in Nursing: A summary of literature published since 2000: Report prepared on behalf of National Education Framework for Cancer Nursing (EDCaN) by Alison Evans Consulting. FitzGerald, M., Walsh, K., & McCutcheon, H. (2001). An Integrative Systematic Review of Indicators of Competence for Practice and Protocol for Validation of Indicators of Competence: Report commissioned by the Queensland Nursing Council. Flanagan, J., Baldwin, S., & Clarke, D. (2000). Work-based learning as a means of developing and assessing nursing competence. Journal of Clinical Nursing, 9, 360-368. Gibson, F., & Soanes, L. (2000). The development of clinical competencies for use on a paediatric oncology nursing course using a nominal group technique. Journal of Clinical Nursing, 9, 459-469. Goodridge, J. M. (2007). Continuing competence programmes for registered nurses: ARNNL Background paper.: Association of Registered Nurses of Newfoundland and Labrador. Gopee, N. (2001). The role of peer assessment and peer review in nursing. British Journal of Nursing, 10(2), 115-121. Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, California: Sage Publications, Inc. Health Practitioners Competence Assurance Act (2003 s118(i)). Health Practitioners Competence Assurance Act (2003, p1 s4(6)). Health Practitioners Competence Assurance Act (2003, s1). Health Practitioners Competence Assurance Act (2003, s11). Health Practitioners Competence Assurance Act (2003, s12). Health Practitioners Competence Assurance Act (2003, s12(4)). Health Practitioners Competence Assurance Act (2003, s27(1)(a)). Hendry, C., Lauder, W., & Roxburgh, M. (2007). The dissemination and uptake of competency frameworks. [Journal]. Journal of Research in Nursing, 12(6), 689-700. Jones, J. (2005). Nurses Remain Atop Honesty and Ethics List Hold substantial lead over other professions. Retrieved from http://www.gallup.com/poll/20254/nurses-remain-atop-honesty-ethics-list.aspx KPMG (2001). KPMG Strategic review of undergraduate nursing education: report to the Nursing Council of New Zealand, May 2001. Wellington, New Zealand: Health, Education and Community Service, KPMG Consulting. McGrath, P., Fox Young, S., Moxham, L., Anastasi, J., Gorman, D., & Tollefson, J. (2006). Collaborative voices: ongoing reflections on nursing competencies. Contemporary Nurse, 22, 46-58. McManus, E. S., & Sieler, P. A. (1998). Freedom to enjoy learning in the 21st century: developing an active learning culture in nursing. Nurse Education Today, 18, 322-332. McMullan, M. (2006). Students perceptions on the use of portfolios in pre-registration nursing education: a questionnaire survey. International Journal of Nursing Studies, 43, 333-343. McMullan, M., Endacott, R., Gray, M. A., Jasper, M., Miller, C. M. L., Scholes, J., et al. (2003). Portfolios and assessment of competence: a review of the literature. Journal of Advanced Nursing, 41(3), 283-294. 6303_nurse_competency copy copy.indd 80

NURSING COUNCIL OF NEW ZEALAND 81 Meretoja, R., Isoaho, H., & Leino-Kilpi, H. (2004). Nurse Competence Scale: development and psychometric testing. Journal of Advanced Nursing, 47(2), 124-133. Miller, S. I., & Fredericks, M. (2006). Mixed-methods and Evaluation research: Trends and issues. Qualitative Health Research, 16, 567-579. Musick, D. W. (1998). Policy analysis in medical education: A structured approach. Medical Education Online, 3(2). Retrieved from http://www.med-ed-online.org/pdf/f0000011.pdf National Nursing and Nursing Education Task Force (2006). National nursing and nursing education task force Final report. Melbourne, Victoria. National Nursing Research Unit (2009). Nursing competence: what are we assessing and how should it be measured? Policy plus evidence, issues and opinions in healthcare, 18, (2). Retrieved from www.kcl. ac.uk/schools/nursing/nnru/policy NCNZ (1994). Strategic Plan 1 April 1994-31 March 1997. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2004a). Continuing Competence Framework. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2004b). Review of guidelines for competence based practising certificates, 2004. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2004c). Draft entry level competencies for the registration of the nurse: Consultation document March 2004. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2004d). Draft continuing practice competencies for the nurse: Consultation document, March 2004. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2005). Report of the Nursing Council of New Zealand for the year ended 31 March 2005. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2006). Continuing Competence Framework. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2008). Request for proposal Evaluation of the continuing competence framework. Wellington, New Zealand: Nursing Council of New Zealand. NCNZ (2009). Nursing Council of New Zealand Definition of Competence. Retrieved 8 November 2009, from <http://www.nursingcouncil.org.nz/contcomp.html> NCNZ (2009). Annual Report 2009. Wellington, New Zealand: Nursing Council of New Zealand NCNZ (2010). Nursing Council of New Zealand - Definition of Practising. Retrieved 9 March 2010, from http:// www.nursingcouncil.org.nz/index.cfm/1,32,html/practising-certificates NCSBN (2009). Assuring competence: a regulatory responsibility. Chicago, USA: National Council of State Boards of Nursing. New Zealand Ministry of Health (2009). Review of the Health Practitioners Competence Assurance Act 2003 Report to the Minister of Health by the Director-General of Health. Wellington. New Zealand Ministry of Health (2010). How Do We Determine if Statutory Regulation is the Most Appropriate Way to Regulate Health Professions? Discussion document. Nurses Regulations (1986). Nursing and Midwifery Council (2008). The PREP Handbook. Retrieved 6 February 2010, from http://www.nmcuk.org/adisplaydocument.aspx?documentid=4340 6303_nurse_competency copy copy.indd 81

82 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK Ovretveit, J. (2000). Evaluating Health Interventions. Buckingham, Philadelphia: Open University Press. Papps, E. (1997). Knowledge, power, and nursing education in New Zealand: A critical analysis of the construction of the nursing identity. Unpublished Doctor of Philosophy, University of Otago, Dunedin. Papps, E. (2002). Nursing in New Zealand. Wellington: Pearson Education New Zealand Ltd. Parlett, M., & Hamilton, D. (1972). Evaluation as Illumination: A new Approach to the Study of Innovatory Programmes. Pearson, A., & FitzGerald, M. (2001). A survey of nurses views on indicators for continuing competence in nursing. Australian Journal of Advanced Nursing, 19(1), 20-26. Pearson, A., FitzGerald, M., & Walsh, K. (2002). Nurses views on competency indicators for Australian nursing. Collegian, 9(1), 36-40. Pearson, A., FitzGerald, M., Walsh, K., & Borbasi, S. (2002). Continuing competence and the regulation of nursing practice. Journal of Nursing Management, 10, 357-364. Sandelowski, M. (2000). Focus on research methods: Combining qualitative and quantitative sampling, data collection and analysis techniques in mixed-methods studies. Research in Nursing & Health, 23, 246-255. Scott Tilley, D. D. (2008). Competency in nursing: A concept analysis. Journal of Continuing Education in Nursing, 39(2), 58-64. Staunton, P., & Chiarella, M. (2008). Nursing and the law (6th Edition ed.). Marrickville, New South Wales: Churchill Livingstone. Storey, L., & Haigh, C. (2002). Portfolios in professional practice. Nurse Education in Practice, 2, 44-48. Thomas, D. R. (2003). A general inductive approach for qualitative data analysis: School of Population Health, University of Auckland, New Zealand. Vernon, R. (2010). Literature Review Competence and the regulation of nurses. Unpublished chapter contributing to PhD Thesis, University of Sydney, Sydney. Vernon, R., Chiarella, M., & Papps, E. (2010). Confidence in competence: legislation and nursing in New Zealand. International Nursing Review, In press. Watson, R., Stimpson, A., Topping, A., & Porock, D. (2002). Clinical competence assessment in nursing: a systematic review of the literature. Journal of Advanced Nursing, 39(5), 421-431. 6303_nurse_competency copy copy.indd 82

NURSING COUNCIL OF NEW ZEALAND 83 Statutes Health Practitioners Competence Assurance (HPCA) Act 2003 Nurses Act 1977 Nurses Registration Act 1901 Nurses Regulations, 1986 Cases Condon, NMT230206JHC in NSW NMB (in press due for publication 2010) Professional Conduct casebook 2nd edition NSW NMB: Sydney Health Care Complaints Commission v Bruce Litchfield,(1997, 41 NSWLR 630) 6303_nurse_competency copy copy.indd 83

84 EVALUATION OF THE CONTINUING COMPETENCE FRAMEWORK 6303_nurse_competency copy copy.indd 84

NURSING COUNCIL OF NEW ZEALAND 85 6303_nurse_competency copy copy.indd 85

www.nursingcouncil.org.nz 6303_nurse_competency copy copy.indd 86