A Delphi study to determine nursing research priorities in Blackwell Publishing Ltd. the North Glasgow University Hospitals NHS Trust and the corresponding evidence base Michelle Kirkwood*, Ann Wales and Anne Wilson, *Librarian for Nursing and Midwifery, Library Services Manager, North Glasgow University Hospitals NHS Trust Library Service, Practice Development Nurse (retired), North Glasgow University Hospitals NHS Trust, Glasgow, UK Abstract The aims of the study are to identify and prioritize research questions of importance to nurses in North Glasgow University Hospitals NHS Trust and to investigate and describe the existing evidence base for the highest priority questions identified. The consensus method known as the Delphi technique was used. Systematic searching of the literature for each question identified from the research priorities enabled the existing evidence base to be characterized. The following priorities were identified as the most important (in descending order of importance): Recruitment and Retention of Nursing Staff; Staff Stress & Morale; Training & Education of Nursing Staff; Infection Control; and Pain Management. The Delphi technique proved a useful method to determine and prioritize research areas in nursing. Introduction The need to expand and develop the research base in nursing, and to improve dissemination and implementation of existing research findings among practising nurses, has reached a new intensity, as nursing seeks to establish its place in the researchbased culture to which the new NHS aspires. Aims and objectives The study was undertaken to: identify and prioritize research questions of importance for nurses in the North Glasgow University Hospitals NHS Trust (from here on referred to as the North Glasgow Trust); investigate and describe the existing evidence base for the highest priority questions identified. Correspondence: Michelle Kirkwood, Librarian for Nursing & Midwifery, North Glasgow University Hospitals NHS Trust Library Service, 10 Alexandra Parade, Glasgow G31 2ER, UK. E-mail: Michelle.kirkwood@northglasgow.scot.nhs.uk Literature review The Delphi technique is a type of survey research that is used to question a panel of experts regarding an issue, and then through sequential structured feedback and response by the same panel, to determine the consensus of the group. 1 The Delphi technique, developed by the RAND corporation in American in the 1950s, has become the method of choice for determining nursing research priorities. 2 A search on CINAHL, using the subject heading Delphi Method or research methodologies as a subject heading combined with the free-text term Delphi, retrieved 526 references, of which 301 described a Delphi study. One hundred and ninety-seven of these studies were specifically within the nursing sphere and 62 of these were studies to ascertain research priorities within specific fields. A similar search to ascertain the prominence of the Delphi technique in nursing research was performed by Beech. 3 CINAHL was searched from 1995 to 2001, using the key words Delphi technique and nursing. Beech 53
54 Delphi of nursing research priorities: Michelle Kirkwood et al. located 145 articles using the Delphi technique to determine clinical practice developments, priorities for research, nurse education and developments, nursing assessment and diagnosis, and aspects of nursing management. This substantial body of literature provided a good reference base for conducting and analysing our project. Methods A three-round Delphi study was used to determine the research priorities for nursing staff in the eight hospitals constituting the North Glasgow Trust. For the purposes of the study, the panel of experts was defined as nurses of any grade working within the North Glasgow Trust. This adheres to the definition from Rudy of the expert as an informed individual. 1 The first round of the Delphi requested nurses of all grades and categories across the North Glasgow Trust to provide five research questions of importance. The 1277 questions from Round 1 were then categorized by a team of three nurses and two librarians into 49 categories, such as: role of the nurse, training and education of staff, infection control. In Round 2, respondents were asked to rank the top 20 categories on a Likert scale of 1 5 where 1 was very low priority and 5 was very high priority, to produce a prioritized list of categories. Round 3 presented respondents from Round 2 with the 20 categories derived from Round 2, together with the overall ranking previously assigned to each category and their own grading. Participants were asked to re-assess the priority they would give each category in the light of their colleagues views. They were also asked to provide one research question for the category they considered to be of highest priority. Three nurses and two information professionals analysed the questions provided for the top five categories and where necessary amalgamated and re-phrased the responses into research questions. Each question was searched in the literature following a search protocol to ascertain the nature of the evidence base. Data analysis Data from Round 1 was analysed by a group of nurses and information professionals in order to collapse the data into categories. Data from Round 2 was analysed to produce a mean score and ranking for each category. Data from Round 3 was again analysed to produce a mean and final rank for each category. Collapsing the responses using a mean has been used in numerous Delphi studies 1,4,5 with ranking the mean being cited as a typical method when prioritization is the objective. 6 Subgroup analysis was undertaken to determine if there was any difference in responses for different demographics. Results Responses Round 1 responses. For Round 1, 2894 nursing staff were sent the initial questionnaire, 458 were returned, 27 of which were spoiled (incomplete or nurse had left the Trust), providing an initial response rate of 15.8. The 421 nurses who did respond to Round 1 produced 1277 questions. These questions were then printed on separate slips of paper. A group of three nurses and two information professionals collapsed the responses into 48 defined categories, such as role of the nurse; patient education; infection control. These 48 categories were grouped into three broad themes: Service Management; Models of Delivering Care; Patient Care. The number of questions provided for each category was calculated as a percentage of total questions within each broad theme. The top 20 categories were selected with proportional representation for each theme. Round 2 responses. In Round 2, the participants were asked to rate each of the top 20 categories from Round 1 on a Likert scale of 1 5. The mean, standard deviation, and rank were calculated for each category. Round 3 responses. In Round 3, the respondents from Round 2 were contacted again. This time they were provided with the overall rating for each category from the results of Round 2 and their own previous response. They were asked to re-rate each question on a Likert scale of 1 5 after considering their colleagues responses. In none of the three rounds was there any follow up. Table 1 shows the top five research priorities.
Delphi of nursing research priorities: Michelle Kirkwood et al. 55 Table 1 Round 3 results:top 5 research priorities in the North Glasgow Trust. Research priorities Rank Mean Std. dev Recruitment and retention of nursing staff 1 4.28 0.88 Staff stress and morale 2 4.27 0.93 Training and education of staff 3 4.24 0.82 Infection control 4 4.20 0.94 Pain management 5 4.16 0.87 Demographics Respondents were spread evenly across the Western Infirmary, Stobhill Hospital and Gartnavel General Hospital sites, with a greater response from the Glasgow Royal Infirmary sites. The smaller hospitals Lightburn, Canniesburn, Drumchapel and Blawarthill were grouped with their parent hospital. Staff Nurses were the group with the highest representation. There was also a fairly even split amongst respondents to Round 3 between the staff nurse grades and below and the charge nurse grade and above. Nurses with more than 20 years experience were the group from which the highest response was obtained. Overall, although the response rate declined at each round of the study, the demographic proportions remained fairly constant. Site, grade, division and years of nursing experience were fairly evenly distributed across the North Glasgow Trust. Subgroup analyses: priorities of junior and senior nursing staff compared. The even demographic split between the junior (staff nurse and below) and the senior grades (charge nurse and above) enables us to ask whether there were significant differences in the way junior and senior staff ranked the categories. Chi squared tests for the top five research priorities showed that there were significant differences in the priorities assigned by junior as compared with senior staff for Pain Management (P = 0. 0.04) and Training and Education of Staff (P = 0.027). In both cases, at least twice as many junior grades considered the categories to be of higher priority than the senior grades. Searching for the evidence As a preliminary indication of the likely nature of the evidence base for the 42 questions, the researchers identified those questions that specifically were concerned with the effectiveness of a health-care intervention. Potentially these would be answerable by randomised controlled trial methodology, regarded as the gold standard for valid clinical research evidence. Twenty out of 42 (48) of the questions were considered to have potential for being answered using this scientific, experimental approach. The remaining questions would be more suitably addressed by observational or qualitative studies. Once the references and data were compiled for each question, a protocol was used to analyse the level of evidence present for each question. A protocol ensures that each question is searched as thoroughly as the next and the results comparable. Figure 1 shows the search protocol used. The number of systematic reviews and meta analyses retrieved using the systematic review and meta analysis filter for CINAHL (NHS Centre for Reviews and Dissemination) was recorded. 7 This number was then calculated as a percentage of the total number of references retrieved (Table 2). This was repeated for trials and other research using the appropriate filter developed by the Cochrane Stroke Group. The number of references not selected by the filters was recorded along with the limits required to bring the number of references retrieved for each individual search down to under 2500. This provides an indication of the character of the evidence base for each priority. Discussion Attrition rate The response rate was disappointing both initially and between rounds. Retention of more nursing staff would have provided a richer choice for prioritization and a more varied selection of questions after Round 3. However the Delphi
56 Delphi of nursing research priorities: Michelle Kirkwood et al. Figure 1 Search protocol Table 2 Summary results of the literature search. Priority Total SR/MA SR/MA trials trials other other remaining remaining Limits Recruitment & retention 2100 93 4.43 186 8.86 318 15.14 1503 71.57 None Stress & morale 2534 96 3.79 405 15.98 586 23.13 1447 57.1 None Training & education 749 26 3.47 115 15.35 85 11.35 523 69.83 None Infection control 5704 300 5.26 1973 34.59 677 11.87 2760 48.39 None Pain management 2884 156 5.41 1202 41.68 630 21.84 890 30.86 None technique dictates a greater level of commitment from participants than a one-off questionnaire. The attrition rate might have been reduced if an initial invitation had gone out to prospective participants. However, this could have biased results even further towards those nurses with particularly strong research interests. Movement of priorities between rounds There was some movement of category rankings between Round 1 and Round 2 (Table 3). This is to be expected as Round 1 serves to generate ideas for research topics, providing a wide divergence of individual opinion 8,9, while Round 2 requested that the participants rank those topics. Although participants may have thought a topic to be of research interest in Round 1, after studying the array of topics provided by fellow participants in Round 2 their allegiance may shift to another topic. Also, the ranking in Round 1 is based solely on the percentage representation for questions within each category, whilst in Round 2 the rank is based on the mean score assigned to each category. A common category could be mentioned frequently Table 3 Movement of priorities between rounds of the Delphi. Priority Rank Round Infection Control 2 Round 1 1 Round 2 4 Round 3 Pain Management 11 Round 1 4 Round 2 5 Round 3 Recruitment and retention of nursing staff 6 Round 1 6 Round 2 1 Round 3 Staff stress and morale 9 Round 1 3 Round 2 2 Round 3 Training and education of staff 4 Round 1 2 Round 2 3 Round 3 in Round 1 but be afforded relatively low priority and scored accordingly in Round 2. Between Round 2 and Round 3 one would expect to see more agreement, therefore less movement in the ranks. Movement would normally be attributable to responders reassessing their views in the light of their colleagues expressed priorities.
Delphi of nursing research priorities: Michelle Kirkwood et al. 57 Generally, this is true with the greatest movement being three places. The exception is Recruitment and Retention of Staff which moved five places in the ranks between Round 2 and Round 3 to become top research priority. Influence of immediate nursing context Recent information has been shown to exercise a strong influence on judgements and opinions in a Delphi study. 6 At the time of the study, the Trust was experiencing a widespread crisis in recruitment and staffing which appears to have influenced responses heavily. Because of the breadth of the population covered in this survey, including all grades, disciplines and specialties, the consensus process necessarily brought to the fore priorities of general interest to all nurses, rather than questions of specific interest to clinical specialties, departments or staff groups. Delphi studies limited to specific groups would help to identify such questions. It is possible that some respondents ranked categories highly because they considered them to be of practical importance due to the day to day delivery of care, rather than research priorities per se. Nature of the nursing evidence base The specific questions provided by nurses were evenly split between the soft subject areas of personnel management and education generally considered to be characterized by management and sociological research methodologies and the harder clinical areas of pain management and infection control, which offer clearly definable and measurable outcomes, and have a strong foundation in the traditional medical and scientific research approaches. Nearly half of the 42 questions potentially could be investigated by the gold standard randomized controlled trial methodology. There is ongoing controversy over how far the evidence-based practice model developed in medicine can be transferred to the nursing field, given the different responsibilities and questions relevant to the two professions. 10,11 It is therefore interesting to see that such a substantial proportion of questions lend themselves to the experimental approach, with the remainder appearing to fit other established research methodologies. The questions defined in this study suggest that there is good reason to support the concept of evidencebased nursing as an achievable reality. The results of the literature searches for the 42 questions revealed that for the soft subject areas (recruitment and retention; staff stress and morale; and training and education of staff) at least one of the filters produced a result. This indicates that there is some form of research base for each question. However (as expected), there were fewer trials available when compared with the harder categories. For the harder subject areas, infection control and pain management, a higher percentage of the articles were research based, in both cases the trial filter produced the highest number of references. Dissemination of research evidence The priorities afforded highest interest by nursing staff of the North Glasgow Trust raise questions concerning the dissemination of research and the pro-active nature of nursing staff in finding the evidence. Pain Management and Infection Control are both areas that have an extensive and rigorous evidence-base. The results of this Delphi study show that North Glasgow Trust staff consider both Pain Management and Infection Control to be important areas for nursing research. Either further research is needed to address specific questions identified by nursing staff (with some such questions having been collected by this survey) or nursing staff are unaware that evidence already exists. This may be due to a lack of dissemination of existing research or failure to seek out the evidence pro-actively. Priorities of junior as opposed to senior staff In the subgroup analysis of the junior and senior staff, almost twice as many junior staff as senior staff considered Pain Management, and Training and Education of Staff of highest importance. There are three possible explanations for this: (i) the senior nurses do not consider the categories to be of high importance; (ii) there is a different perception of the evidence-base for these categories; (iii) there is a specific need for further research in these areas which is particularly relevant to the staff who work on the front line.
58 Delphi of nursing research priorities: Michelle Kirkwood et al. Years of experience of responders In the demographics of the study, it was noted that the dominant group of responders (44 in the Round 3) had over 20 years experience. A recent survey on the age of the nurse population of the North Glasgow Trust found 72 of nurses were aged between 25 and 45 years of age. Within this group the highest percentage (21.38) falls within the 35 40 years range. This older nurse population may account for the high number of responses from the > 20 years experience group. Unfortunately, the population survey provided only details of age groups and did not link this to number of years nursing experience. This group may have participated more than any other in the Delphi study, either because they have been neglected in the past and have taken this opportunity to have their say, or because, as a group, they hold strong feelings on nursing research. Conclusions It is unclear the extent to which the results of this Delphi study may reflect nursing priorities for research, not only in the North Glasgow Trust, but also in the NHS as a whole. This issue merits further investigation. Although responses may reflect areas of general concern to nursing staff rather than specifically areas requiring research, the results provide an insight into nursing today. They also highlight the need, not only to pursue the development of an evidence base for nursing, but also to disseminate information on the existing evidence. Characterizing the evidence base through searching CINAHL and using evidence-based filters provided an objective insight into the nature of the evidence available. A more thorough search using multiple databases and subject hedges would have yielded further insights into specific topic areas. However, the protocol was a useful means for identifying the overall characteristics of the evidence. The Delphi technique was found to be a useful method to arrive at consensus amongst a large group of nurses and provided a powerful tool for identifying research priorities. However, to produce research priorities of a more specific nature, and also to reduce the attrition rate, a smaller Delphi study within a nursing specialty would be more appropriate. Acknowledgements LINC (now HeLICON) Panel: Research in the Workplace Award 2000 for funding and support. For assistance and support; Mark Cooper, Research Practitioner, A & E, Glasgow Royal Infirmary and Patricia O Gorman, Clinical Nurse Specialist, Palliative Care, Glasgow Royal Infirmary. For participation in the project: the nursing staff of North Glasgow University Hospitals NHS Trust. References 1Rudy, S. A review of Delphi surveys conducted to establish research priorities by specialty nursing organizations from 1985 1995. ORL-Head and Neck Nursing 1996, 14(2),16 24. 2Forte, P. S., Ritz, L. & Balestracci, D. Identifying nursing research priorities in a newly merged healthcare system. JONA 1997, 27(6), 51 5. 3 Beech, B. The Delphi approach: recent applications in health care. Nurse Researcher 2001, 8(4), 38 52. 4 Bartu, N., Nelson, M., McGowan, S. & Robertson, J. A Delphi survey of clinical nursing research priorities in Western Australia. Australian Journal of Advanced Nursing 1991, 8(3), 29 33. 5 Fitzpatrick, E., Sullivan, J., Smith, A., Mucowski, D. et al. Clinical nursing research priorities: a Delphi study. Clinical Nurse Specialist 1991, 5, 94 9. 6 Murphy, M. K., Black, N. A., Lamping, D. L., McKee, C. M., et al. Consensus development methods and their use in clinical guideline development. Health Technology Assessment 1998, 2(3), whole issue. 7 NHS Centre for Reviews and Dissemination. CINAHL search strategy for systematic reviews and meta analyses. Search Strategies to Identify Reviews and Meta-analysis in MEDLINE and CINAHL. Available from: http://www1.york.ac.uk/institute/crd/search.htm (last updatedapril 2002) (accessed 13 November 2002). 8 Hasson, F., Kenney, S. & McKenna, H. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing 2000, 32, 1008 15. 9 McKenna, H. P. The Delphi technique: a worthwhile research approach in nursing? Journal of Advanced Nursing 1994, 19, 1221 5. 10 Bonell, C. Evidence-based nursing: a stereotyped view of quantitative and experimental research could work against professional autonomy and authority. Journal of Advanced Nursing 1999, 30, 18 23. 11 Closs, S. J. & Cheater, F. M. Evidence for nursing practice: a clarification of the issues. Journal of Advanced Nursing 1999, 30, 10 7.