Devising and establishing the face and content validity of explicit criteria of consultation competence for UK secondary care nurses

Nurse Education Today (2004) 24, 180 187 Nurse Education Today intl.elsevierhealth.com/journals/nedt Devising and establishing the face and content validity of explicit criteria of consultation competence for UK secondary care nurses Sarah A. Redsell a, *, Marianne Lennon a, Adrian M. Hastings b, Robin C. Fraser a a Division of General Practice and Primary Health Care, Department of Health Sciences, University of Leicester, Gwendolen Road, Leicester LE5 4PW, UK b Department of Medical and Social Care Education, University of Leicester, Gwendolen Road, Leicester LE5 4PW, UK Accepted 26 November 2003 KEYWORDS Nurse consultants; Specialist nurses; Nurses; Competence; Consultation; Assessment; CAIIN Summary Aim: The aims of this study were (a) to devise a set of prioritised criteria of consultation competence for UK secondary care nurses, and (b) to determine their face and content validity. Method: The criteria of consultation competence as contained in the Consultation Assessment and Improvement Instrument (CAIIN) for primary care nurses were adapted as a result of focus group discussions and observation of videotaped consultations with nurses working in secondary care. The amended criteria were sent to a sample of nurse consultants, specialist nurses and nurse practitioners (n ¼ 394) to determine their face and content validity. Results: Support for the seven categories of consultation competence varied from 96% to 99% and for the 37 component competences from 94% to 99%. There was no consensus for alternative or additional categories or components. 87% of respondents strongly agreed or agreed that the categories of consultation competence should be prioritised and 63% strongly agreed or agreed with the suggested weightings. Conclusion: We have devised prioritised criteria of consultation competence of UK secondary care nurses and established their face and content validity. This can now facilitate the assessment and improvement of the consultation competence of secondary care nurses for both formative and regulatory purposes. c 2003 Elsevier Ltd. All rights reserved. Introduction * Corresponding author. Tel.: +44-116-258-4873; fax: +44-116- 258-4982. E-mail addresses: sr59@le.ac.uk, sr59@leicester.ac.uk (S.A. Redsell). Over the past decade there has been a huge expansion and diversification of nursing roles within the NHS (Read et al., 1998) and new roles have emerged such as specialist nurse and nurse consultant. Increasingly patients initial contact with the NHS is a consultation with a nurse, rather than a 0260-6917/$ - see front matter c 2003 Elsevier Ltd. All rights reserved. doi:10.1016/j.nedt.2003.11.006

Explicit criteria of consultation competence for UK secondary care nurses 181 doctor. Nurses are also leading health care provision for some patient categories in walk-in centres, NHS Direct, A&E and minor illness units and in general practice. In these clinical areas nurses are required to conduct initial assessments and encounter a wide range of problems, make referral decisions and manage patients illnesses. In addition nurses are working as autonomous practitioners, providing care for patients with complex problems associated with chronic illness. However, concern has been expressed that many nurses lack the appropriate skills and preparation for these new roles (Read et al., 1998) and the question of how to ensure that nurses are competent to perform these new roles has not been adequately addressed. A recent systematic review of the literature revealed that most of the current methods used to define or measure clinical competence in nursing have not been developed systematically and insufficient attention has been paid to their reliability and validity (Watson et al., 2002). The first step to improving nurses competence in the consultation setting is to determine the criteria against which their performance can be judged. We have identified and validated explicit criteria of consultation competence for primary care nurses and incorporated these in the Consultation Assessment and Improvement Instrument for Nurses (CAIIN) (Redsell et al., 2003). The CAIIN is now being used to develop the consultation skills of nurses working in first contact services (Department of Health, 2002). The criteria of consultation competence in the CAIIN were derived from those in the Leicester Assessment Package (LAP), which was developed primarily for use with general practitioners (Fraser et al., 1994a,b). Although almost all the consultation competences are generic and, therefore, common to both medical and nursing practice, several were modified for primary care nurse consultations. Consequently, it was anticipated that this would also be necessary when adapting the primary care CAIIN for nurses working in secondary care. A study with the following aims was therefore carried out: To devise a set of prioritised criteria of consultation competence for UK secondary care nurses, and To determine their face 1 and content 2 validity. 1 The extent to which the assessment instrument subjectively appears to be measuring what it is supposed to measure. 2 The extent to which the instrument includes a representative sample of the content of a construct. Method Focus groups and video recording The project involved adapting the criteria of consultation competence in the primary care CAIIN for nurses 3 working in secondary care. To facilitate this task eight focus groups were conducted with volunteer nurses from a wide variety of speciality areas. The nurses were asked to scrutinise the CAIIN criteria and make suggestions for their continuing inclusion, removal and/or amendment or whether new criteria should be derived. In addition 18 video recordings of seven nurses in consultation with real patients were scrutinised by SR, ML and AH in order to provide insights into the suitability of the CAIIN criteria with particular reference to interviewing, care planning, health promotion and relationship with patients. The nurses were drawn from two hospital sites in the UK serving differing populations. One covers an inner city high ethnic minority population, but also includes rural areas and the other a mixed rural and urban area. National validation survey The target group for the validation survey were nurses working as nurse practitioners, specialist nurses or nurse consultants in secondary care throughout the UK. The criteria of consultation competence, amended as a result of the focus group discussion and observation of video recorded consultations, were incorporated into a questionnaire which was sent to the identified sample of nurses (see Fig. 1) together with an explanatory letter. The nurses were asked to indicate their level of agreement with the proposed consultation categories and component competences using a four-point scale (strongly agree, agree, disagree or strongly disagree). They were given the opportunity to reject any of the proposed categories and/or components, to suggest additional categories and/or components and to provide free text comments. The nurses were also invited to apply priority weightings to the consultation categories by indicating their level of agreement for our suggested weightings or to suggest alternatives. Non-respondents were followedup six weeks later by post. 3 The term nurse describes all the categories of nurses who participated in this project (nurse practitioners, specialist nurses and nurse consultants).

182 S.A. Redsell et al. TARGET GROUP (SAMPLING FRAME) All nurse practitioners, specialist nurses and nurse consultants working as practitioners or educators in secondary care in the UK PROCEDURE TO OBTAIN SAMPLE 65 educational institutions written to PROCEDURE TO OBTAIN SAMPLE Foundation of Nursing Studies had a database of nurse practitioners, specialist nurses and nurse consultants FONS facilitated the sample acquisition by sending out a letter on our behalf. FONS database did not discriminate between primary and secondary care 1 PROCEDURE TO OBTAIN SAMPLE 148 secondary care institutions contacted 9 Directors of Nursing replied SAMPLE 55 educators agreed to participate SAMPLE 640 nurses in target group on FONS database SAMPLE 43 nurses in target group volunteered to participate FINAL RESPONSE RATE 35/55 (63%) FINAL RESPONSE RATE 359/683 (52%) 2 1. Participants were invited to return the questionnaire if they were currently working in secondary care. 43 advised us that they were not in this target group (leaving a total of 640). It was not possible to differentiate the non-respondents from those who did not fulfil the target criteria (or had left their post). 2. Due to data protection limitations, it was not possible to discriminate between those nurses who responded via the invitation from the FONS database and those who responded via the invitation from the Directors of Nursing. Figure 1 The sampling methods and response rates. It was not possible to make sample size calculations since there is no published register of nurse practitioners, specialist nurses and nurse consultants currently employed in the NHS. The maximum number of nurse consultants has been estimated at around 1000 (Department of Health, 2000) but many posts have not yet been filled. Given the high level of agreement and low standard deviation obtained in the validation exercise for the primary care CAIIN (Redsell et al., 2003), it was anticipated that a sample size of 100 nurses would be adequate to obtain a minimum of 80% agreement for the proposed criteria. However, the difficulties obtaining a sample and the response rate of 62% achieved in the validation exercise in the primary care study suggested that over-sampling would be required. Research governance Approval was sought and obtained from two local research ethics committees (LREC). Written informed consent was obtained from both the nurses and patients in order to video record the consultations and the General Medical Council guidelines were followed. Participants were assured that all information provided would be treated in complete

Explicit criteria of consultation competence for UK secondary care nurses 183 confidence and that all video recordings would be stored in a locked cupboard and destroyed on completion of the project. Results Four focus groups were conducted at each site. The numbers attending each session varied from 5 to 19, with a core group of five to six nurses attending most sessions. Seven of the regular attendees agreed to be videotaped in consultation with 18 patients. Nurses from a variety of specialist areas volunteered to participate. The speciality areas were GU medicine, respiratory, diabetes services, gastroenterology, cardio-thoracic, occupational therapy, incontinence services and colorectal. As a result of the above, no criteria were added or removed from the CAIIN, but adjustments were made to the components and the terminology used was made more appropriate for a secondary care setting. The major changes and additions are outlined in Table 1. The focus group nurses were keen to adopt an open approach to their consultations and requested Table 1 Major changes and new components for the secondary care caiin and rationale. Criterion/component in primary care New or omitted component Rationale CAIIN Interviewing Demonstrates an ability to formulate open questions Focus group nurses wanted to avoid dominating the consultation with their own agenda. Examination, diagnostic testing and practical procedures Performs near patient testing correctly Care planning/patient management Provides appropriate reassurance and explanation Checks patients /clients level of understanding Problem solving Seeks relevant clinical signs Component removed Reaches a shared understanding about the problem with the patient Seeks relevant clinical signs and makes appropriate use of clinical tests Practice not familiar to focus groups nurses. Two components collapsed in order to emphasise the importance of the two way process. Two components collapsed and re-worded in order to fit with nurses practice. Makes appropriate use of near patient testing Uses protocols as appropriate Component removed Not considered part of problem solving. Components in care planning/patient management cover this. Identifies patients /clients reason(s) for attendance and associated concerns and expectations Record keeping New component Makes an appropriate record of the consultation Explores patient s ideas, concerns and expectations about their problem(s) Provides the names(s), dose and quantity of drug(s) prescribed to the patient together with any special precautions Makes an appropriate and legible record of the consultation Focus group nurses wanted to remove the association with attendance as patients might already be in hospital. Preferred to use explores. Component added to allow for nurse prescribing. Focus groups requested legible made explicit.

184 S.A. Redsell et al. Table 2 Levels of agreement for proposed consultation categories and distribution of agreed weightings. Consultation category Strongly agreed/agreed category should be included % (95% CI) a Distribution of agreed weightings (%) Care planning/patient management 99 20 Record keeping 99 10 Interviewing 98 20 Problem solving 98 15 Examination, diagnostic testing and 97 10 practical procedures Health promotion/disease prevention 97 10 Behaviour/relationship with patient 96 15 that a component be added which explicitly refers to demonstrates an ability to formulate open questions. From the videotape observations the researchers also noted that many nurses had difficulty with the organisation of the consultation. In particular, there were occasions where the patient and nurse did not reach a shared understanding of the problem and therefore the nurses found it difficult to negotiate a management plan. In light of this it was agreed to include reaches a shared understanding about the problem as an explicit component. The focus group nurses were not familiar with the term near patient testing and therefore any reference to this was removed from the secondary care CAIIN. A component to describe nurse prescribing provides the names(s), dose and quantity of drug(s) prescribed to the patient together with any special precautions was added to the record keeping criteria. Finally, a number of changes were made to the language used to describe the criteria and components to make them more suitable for a secondary care context. For example, the word patient was applied throughout rather than patient/client, which is used in the primary care CAIIN. Fig. 1 indicates the sampling frame and response rates for the validation survey. There were respective response rates of 63% (35/55) from the nurse educators and 52% (359/683) from the nurse practitioners, specialist nurses and nurse consultants. Tables 2 and 3 indicate that the levels of support for the inclusion of the seven categories of consultation competence varied from 96% to 99% and for the 37 component competences from 94% to 99%. Eighty-seven percent of respondents strongly agreed or agreed that the categories of consultation competence should be prioritised and 63% strongly agreed or agreed with the suggested weightings (see Table 2). Of those who disagreed there was no consensus regarding alternative allocation of priority weighting. Discussion We have now established the face and content validity of a set of explicit and prioritised criteria of consultation competence for secondary care nurses. The levels of support for the seven proposed categories (96 99%) and 37 component competences (94 99%) were very high. There was also a high level of support for the principle of applying priority weighting to the established categories (87%) and majority support (63%) for our proposed weightings with no consensus amongst those who disagreed. Although the focus group discussions and videotaping revealed differences between nurses working in primary and secondary care, our study shows that there is a substantial degree of overlap in the agreed consultation competences of primary and secondary care nurses. Apart from minor differences these consultation competences are generic for nurses who consult with patients on a one-to-one basis. These criteria have now been incorporated in a CAIIN pack for secondary care nurses (CAIIN2) in the same way as for the primary care nurses (CAIIN1) (Redsell et al., 2003). CAIIN2 can be used for developing and supporting new and changing nurses roles, in particular, those of nurse consultants and nurse specialists, and to evaluate nurses undertaking extended roles such as nurse prescribing. It could be used to enable student nurses to develop levels of consultation capability and in continuing professional development (including clinical supervision). Once reliability has been established it may be

Explicit criteria of consultation competence for UK secondary care nurses 185 Table 3 Levels of agreement for proposed component competences. Component competence Total in favour % (95% CI) Interviewing Demonstrates an ability to formulate open questions 99 Enables patient to explain situation/problem fully 99 Recognises patients verbal and non-verbal cues 99 Listens attentively 98 Considers physical, social and psychological factors as appropriate 99 Phrases questions simply and clearly 99 Seeks clarification of words used by patient as appropriate 98 Uses silence appropriately 98 Puts patient at ease 97 Demonstrates a well-organised approach to information gathering 97 Examination, diagnostic testing and practical procedures Elicits physical signs correctly and sensitively 99 Performs technical procedures in a competent and sensitive manner 98 Uses instruments in a competent and sensitive manner 97 Care planning and patient management Reaches a shared understanding about the problem with the patient 99 Uses clear and understandable language 99 Arranges appropriate follow-up 99 Negotiates care plans with patient 98 Formulates and follows appropriate care plans 98 Educates patients appropriately in practical procedures 98 Makes discriminating use of referral investigations and drug treatment 95 Problem solving Accesses relevant information from patients records 99 Explores patients ideas, concerns and expectations about their problem(s) 99 Elicits relevant information from patient 99 Seeks relevant clinical signs and makes appropriate use of clinical tests 99 Correctly interprets information gathered 99 Applies clinical knowledge appropriately in the identification and management of the patients 99 problem Recognises limits of personal competence and acts accordingly 99 Behaviour/relationship with patients Conveys sensitivity to the needs of patient 98 Maintains friendly but professional relationships with the patient 96 Is able to use professional relationship to achieve mutual agreement with care plan 93 Health promotion/disease prevention Acts on appropriate opportunities for health promotion and disease prevention 98 Provides appropriate explanation to the patient for preventative initiatives 98 suggested Works in partnership with the patient to encourage the adoption of a healthier lifestyle 94 Record keeping Makes an appropriate and legible record of the consultation 99 Records care plan to include advice and follow-up arrangements as appropriate 99 Enters results of measurements in records 98 Provides the names(s), dose and quantity of drug(s) prescribed to the patient 95 together with any special precautions used to facilitate the identification and remedy of poor performance and for professional selfregulation. There was a high level of agreement amongst the nurses who participated in the validation exercise and we acknowledge that the methodology

186 S.A. Redsell et al. used may have contributed to this. Nurses who participated in both the focus groups and the validation survey were asked to adapt the criteria in an existing instrument (CAIIN1). The alternative would be to develop an entirely new instrument but since the construct validity of CAIIN1 (Redsell et al., 2003) was so high this approach would be difficult to justify. We encountered difficulties in obtaining participants for all aspects of this project. Initially there was a high level of enthusiasm for the project from nurses at both sites and attendance at introductory sessions was high. However, the numbers attending the focus groups varied considerably from week to week. Non-attendance was mainly attributed to workforce pressures and this was particularly apparent during the school half-term week. We also experienced difficulty in recruiting nurses willing to allow their consultations to be videotaped with the result that the number of consultations recorded was fewer than intended (18v36). A number of reasons for this are proposed. Firstly, videotaping has not been as widely used in nurse education as it has in medicine and some of the nurses may have been reluctant because this method of learning was unfamiliar. Secondly, despite viewing the protocol and our reassurances, many of the nurses were suspicious about what we would use the videotapes for. Some of the nurses felt others would judge their performance negatively if it was recorded and they found the idea of learning using this approach daunting. Nurses are concerned about their consultations being observed via the use of videotape and other methods need to be explored in future research. We also experienced difficulty in determining a representative sample of nurses (nurse practitioners, specialist nurses and nurse consultants) for the validation exercise as no single database exists which specifically identifies these groups. The method of contacting all Directors of Nursing (practice) and Heads of Departments (education) to request they invite their staff to help us with the project yielded few participants. We were therefore obliged to utilise the services of the Foundation of Nursing Studies (FONS) database of nurse practitioners, specialist nurses and nurse consultants, which although not comprehensive is probably the most up to date record available. As a result our sample was purposive rather than a representative and until comprehensive and complete databases are available this is likely to be the only way of sampling. Further research is required to ascertain whether these consultation competences are generic for other health care professionals within the multi-professional team. Conclusion We have adapted the primary care CAIIN for nurses working in secondary care using a two-phase process. The use of focus groups and the observation of video taped consultations, enabled us to make appropriate changes to the CAIIN1 components to reflect the secondary care context. Nurse practitioners, specialist nurses and nurse consultants have endorsed our seven proposed categories of consultation competence, the associated component competences and our suggested weightings. The CAIIN2 criteria and components can now be used as the basis for assessing and improving the consultation skills of secondary care nurses. Acknowledgements We wish to thank the participating nurse practitioners, nurse consultants and specialist nurses, the intermediaries working in secondary care and University Departments of Nursing who passed on information about the study on our behalf and the Foundation of Nursing Studies (FONS). Trent Region Research and Development Scheme funded this study. References Department of Health, 2000. The National Plan, The Stationary Office, London. Department of Health, 2002. The First Contact Programme Chief Nursing Officer Bulletin Special Edition, July (www.doh. gov.uk/cno/pctbulletinspecialjuly02.htm) (accessed November 2003). Fraser, R.C., McKinley, R.K., Mulholland, H., 1994a. Consultation competence in general practice: establishing the face validity of prioritised criteria in the Leicester Assessment Package. Br. J. Gen. Pract. 104, 109 113. Fraser, R.C., McKinley, R.K., Mulholland, H., 1994b. Consultation competence in general practice: testing the reliability of the Leicester Assessment Package. Br. J. Gen. Pract. 44, 293 296. General Medical Council, 2002. Making and using visual and audio recordings of patients (http://www.gmc-uk.org/standards) (accessed November 2003). Read, S., Lloyd-Jones, M., Collins, K., McDonnell, A., Jones, R., Doyal, L., Cameron, A., Masterson, A., Dowling, A., Vaughan, B., Furlong, S., Scholes, J., 1998. Exploring New Roles in Practice: Implications of developments within the clinical team (ENRiP) University of Sheffield (http://www.snm. shef.ac.uk/research/enrip/enrip.pdf) (accessed November 2003).

Explicit criteria of consultation competence for UK secondary care nurses 187 Redsell, S.A., Hastings, A., Cheater, F.M., Fraser, R.C., 2003. Devising and establishing the face and content validity of explicit criteria of consultation competence in UK primary care nurses (CAIIN). Nurse Education Today 23 (4), 299 306. Watson, R., Stimpson, A., Stimpson, A., Topping, A., Porock, D., 2002. Clinical competence assessment in nursing: a systematic review of the literature. J. Adv. Nursing 39 (5), 421 431.