Process evaluation of a nursing intervention to develop a research culture among orthopaedic nurses A triangulation convergence model Protocol Research team: Connie Bøttcher Berthelsen Bibi Hølge-Hazelton
Background The tradition of conducting nursing research in Denmark has evolved over the past two decades and the focus from the government on funding and academic positions as well as the facilitation of academic supervision from the universities continue to grow (Poulsen& Lindhardt 2013). Research in nursing is still very limited in the hospitals, compared to the production of research in the universities especially in the orthopaedic departments. In order to support the development of research culture among orthopaedic nurses we conducted a cross-sectional survey using a 24-item questionnaire to explore the knowledge, interest and motivation for nursing research among nurses at an orthopaedic department in a Danish Regional Hospital (Berthelsen& Hølge-Hazelton, in process). All nurses employed at the orthopedic department at the hospital (n=87) was recruited through convenience sampling to participate in the study. 43 orthopaedic nurses completed the questionnaire, giving a response rate of 49.43 %. 32 nurses (65%) were assessed highly motivated and interested to increase their knowledge of research and to participate in nursing research studies. On the basis of these results we initiated an intervention to support the development a research culture among orthopaedic nurses for the 32 motivated nurses, consisting of six teaching sessions of nursing research in general and six nursing research studies of the participants choice. However, only a mean average of six nurses participated during the teaching sessions that lasted from April to June of 2013. To establish understanding of the nurses lack of participation in the teaching sessions, we decided to conduct a process evaluation of the intervention to explore the barriers perceived by nurses during participation and to improve the stages for further implementation. Process evaluations aim to examine how the intervention is implemented, the contextual factors that affect the intervention and the views of the participants (Oakley et al. 2006). The method is often used in randomised controlled trials to evaluate the effectiveness of the intervention stages and to interpret the outcome results. The UK Medical Research Council (MRC) guideline for the evaluation of complex interventions explained an essential need for understanding the problems of integration of interventions into healthcare settings (Campbell et al. 2007). They propose that barriers and facilitators to implementation should be investigated to understand the difficulties that may appear during the intervention. 2
Aims and objectives In this study we aim to conduct a process evaluation of the intervention of supporting and developing research culture among orthopaedic nurses through two arms: 1) By describing the orthopaedic nurses actual exposure to the intervention. The research question is: - How did the orthopaedic nurses evaluate the teaching sessions after participation? 2) By describing the orthopaedic nurses experiences and barriers during the intervention. The research questions are: - Which barriers did the orthopaedic nurses perceive for participation in the teaching sessions? - How did the orthopaedic nurses experience the intervention and the relevance of the intervention? Methods Design A convergence model according to Creswell and Plano Clark s typologies of triangulation mixed methods design (2007) will be used to combine quantitative and qualitative data during the interpretation phase. Using the convergence model, we will concurrently collect and analyse quantitative and qualitative data on orthopaedic nurses actual expose to and experience of the intervention. The quantitative results and qualitative findings will then be converged, by comparing and contrasting the different results, during the interpretation (Figure 1). The model is appropriate when researchers want to validate or compare results and end up with well-substantiated conclusions about a single phenomenon (Creswell& Plano Clark 2007). 3
Figure 1: Triangulation design: Convergence model (Creswell & Plano Clark 2007) QUAN Data collection QUAL Data collection QUAN Data analysis QUAL Data analysis QUAN Results QUAL Findings Compare and contrast Interpretation QUAN + QUAL Participants The 32 orthopaedic nurses, who responded positively to having a high interest in research and being interested and motivated for developing their research skills and knowledge in the cross sectional survey (Berthelsen & Hølge-Hazelton, in process) were invited to participate in the intervention. 40% of the participants had a bachelor degree in nursing and 19% were currently studying for a bachelor or a master degree. 63% of the participants were between 40 and 65 years of age. The sample consisted of 31 females and 1 male. The intervention We initiated an intervention of supporting and developing research culture among the 32 highly motivated orthopaedic nurses on the basis of the results of the survey regarding the nurses specific wishes for learning. The aim of the intervention was take point of departure in the orthopaedic nurses interest to systematically and inspirationally develop their theoretical and practical knowledge of nursing research. The intervention planned The intervention was planned to consist of six teaching sessions of nursing research in general, followed by six nursing research studies. These six studies were chosen by the participants for them to perform on the basis of what they learned during the teaching sessions. The teaching sessions was scheduled to every other Friday from 13 to 15 o clock. The scheduled time frame was chosen through conversations with the head nurses as well as the four leading nurses of each unit, because we wanted the sessions to be available for as many of the participants as possible. Each session was announced on the previous Monday with information 4
about content of the session. The six teaching sessions were planned to consist of nursing research in general, qualitative and quantitative design and methodology, and data collection and analysis in qualitative and quantitative studies (Table 1). The subject was chosen in order to cover the basic areas of nursing research however in a very short amount of time. We planned to teach the participants subsequently during their research studies in the department as question might arise during performing research. The six research studies were established through the orthopaedic nurses requests in the survey about their research ideas from practice. The studies have not yet been conducted due to intervention failure. The intervention performed The six teaching sessions were initiated as planned however major changes were made concurrently (table 1). The first session concerned the current intervention, why nurses need to be involved in research, nursing research in general, and the differences between qualitative and quantitative research. After the session the participants were asked to evaluate the teaching through a LEARN-schedule (Hølge-Hazelton& Tulinius 2008). The participants comments to the first session were that they needed more discussions and dialog with the teacher rather than a one-way communication. They also expressed needs for discussing their research studies, which were to be conducted after the six teaching lessons. In order to maintain focus and interest by the participants, we chose to change the second teaching sessions from addressing methodology and design to a discussion of the six research studies, the content and participant selection for each study. The session of theoretical and methodological approaches was divides in the session on qualitative and quantitative designs, respectively. The third teaching session (3a) addressed qualitative designs and methods. Ten participants had accepted the invitation however only two participants showed up. The third session (3b) was moved to a different date where four participants showed up. Their evaluations were still positive towards the teaching sessions and the subjects planned. The forth session was planned to address data collection and analysis in qualitative studies. One participant of the 32 answered the invitation however the session was cancelled. Due to lack of participating nurses we chose to stop the intervention to evaluate the process of what to do differently next time. 5
Table 1: The differences in the intervention planned and performed Teaching session Intervention as planned Intervention as performed Participants 1 Introduction to intervention, the projects and nursing research in general. Introduction to intervention, the projects and nursing research in general. 10 2 Theoretical and methodological approaches Discussion of the six research studies, content and participant selection 11 3a Qualitative designs and methods Cancelled 2 3b Qualitative designs and methods Qualitative designs and methods 4 4 Data collection and analysis in qualitative Cancelled 1 studies 5 Quantitative designs and methods Not conducted - 6 Data collection and analysis in quantitative studies Not conducted - Data collection The process evaluation framework of Hulscher and colleagues (2002) will be used to describe the actual exposure and to evaluate the experience of those exposed to the intervention. Usually process evaluation is used to evaluate the effectiveness of the implementation (Hulscher et al. 2002). In this case, we used the process evaluation framework to increase our knowledge of why most of the orthopaedic nurses referred from participating in the intervention, even though the results of our survey showed they were highly motivated, rather than assessing how much they learned. Actual exposure to the intervention To describe the participants actual exposure to the intervention, we want to explore how the participants evaluated the teaching sessions. After each teaching session the participants present were asked to use five minutes to evaluate the teaching through LEARN-schedules (Hølge- Hazelton & Tulinius 2008). The LEARN-schedules was developed focusing on evaluation of the process of learning. The LEARN-schedule consisted of eight questions. The first part allowed for open-ended responses to five questions: 1) What was the most important thing I learned today? 2) What is still not clear? 3) What would I like to learn more about? 4) Which knowledge will I use in the future? 5) How did I experience today s teaching session? In the second part of the LEARNschedule answers could be given on a 4-point Likert scale. The sixth question was assessed on a 4-6
point Likert scale from a great extent to no extent : 6) Should research become a central part of my workday? And the seventh and eighth question was assessed on a 4-point Likert scale from high to none : 7) How is my interest in research? 8) How is my motivation for research? Experiences and barriers during the intervention We also want to explore the barriers perceived by the orthopaedic nurses for participation in the teaching sessions and how they experienced the intervention and the changes. The barriers will be grouped into three categories: Relevance of teaching sessions, internal barriers, and external barriers. A newly developed 21-item questionnaire, inspired by The BARRIERS to Research Utilization Scale by Funk and colleagues (1991) will be used to collect data about the barriers perceived by the orthopaedic nurses for participation in the teaching sessions. The questions are developed around the three categories: Relevance of teaching sessions (question 1-8), internal barriers (question 9-14), and external barriers (question 15-19). The answers to question 1-19 will be given on a 4- point Likert scale (1 = to no extent, 2 = to a little extent, 3 = to a moderate extent, 4 = to a great extent). In addition the participants are invited to rank the three barriers they perceive as the greatest (question 20) and to list the factors they perceive as facilitators for research utilization (question 21). Two focus group sessions will be conducted to elaborate the preliminary results of the questionnaire by focusing on the barriers of participation and to explore the orthopaedic nurses to the intervention and changes. The participants for the first session will be 5-7 orthopaedic nurses who were present during the teaching sessions and the second session will consist of 5-7 orthopaedic nurses who were never present. The focus group sessions will be conducted by the clinical development nurse of the orthopaedic department because of my bias presence. A nurse from another orthopaedic department in Copenhagen will be present during the sessions to write field notes. The sessions will be tape-recorded and transcribed verbatim. Data analysis We will use descriptive analysis to analyse the data regarding the second part of the LEARNschedules and the responses to the questionnaire, using STATA (12.0) software. The data from the focus group sessions will be analysed through manifest content analysis (Berg 2004, Catanzaro 7
1988), to identify the three categories of: Relevance of teaching sessions, internal barriers, and external barriers, in the interview transcripts. The quantitative and qualitative data sets will then be compared and contrasted during interpretation and presented as results. The data from the open-ended questions in the first part of the LEARN- schedule will be not be analysed but used to illustrate examples. Ethical considerations The participants will be informed about the evaluation process in writing through the department newsletter and orally before receiving the questionnaires and at the focus group sessions. There will be applied for approval from the Danish Data Protection Agency and the National Committee on Health Research Ethics. The study will be performed according to the Helsinki Declaration. Limitations of mixed method research Mixed methods research has much to offer health and social sciences by bridging the gap between quantitative and qualitative positions (Doyle et al. 2009). However, critics focus on the incompatibility of the two research paradigm because of their strongly dichotomy of epistemological and ontological world views (Creswell& Plano Clark 2007). We chose a mixed methods design since it provides a greater repertoire of tools to meet the aims and objectives of our study. By following up on quantitative findings from our questionnaires of the barriers the participants experienced by conducting focus group sessions with a sample of those participants, we gain a larger understanding of the nurses barriers and experiences of the intervention. Relevance for practice and further research The knowledge we derive from this study will be used to further develop the intervention of developing a research culture among orthopaedic nurses. Time frame The study will be conducted from September to November 2013. The questionnaire will be handed out from the 2 nd 6 th of September and the focus group sessions will be conducted the 16 th and 17 th of September. Data analysis will take place in October. 8
Publication The study will be submitted to the Journal of Clinical Nursing in November. Furthermore the study will be a part of a book chapter describing the barriers of nurses research utilization. The book is scheduled to be published I March 2014. Budget There is no additional cost of conducting this study. Data collection and analysis will be performed by the authors. 9
References Berg B (2004) Qualitative Research Methods for the Social Sciences. Pearson Education inc. Boston. Berthelsen CB & Hølge-Hazelton B (In process) Orthopaedic nurses knowledge, interest and motivation for clinical nursing research - A cross-sectional survey. Campbell N, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, Guthrie B, Lester H, Wilson P & Kinmonth AL (2007) Designing and evaluating complex interventions to improve health care. British Medical Journal 334, 455-59. Catanzaro M (1988) Using qualitative analytic techniques. In:Woods NF, Catanzaro M. (eds.), Nursing Research. Theory and Practice. The CW Mosby Company, USA. Creswell JW & Plano Clark VL (2007) Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: SAGE. Doyle L, Brady AM & Byrne G (2009) An overview of mixed methods research. Journal of Research in Nursing 14, 175-185. Funk SG, Champagne MT, Wiese RA & Tornquist EM (1991) BARRIERS: The barriers to research utilzation scale. Applied Nursing Research 4, 39-45. Hulscher MEJL, Laurant MGH & Grol, RPTM (2003) Process evaluation on quality improvement interventions. Quality Safety Health Care 12, 40-46. Hølge-Hazelton B, Tulinius C (2008). Forskningsenheden for Almen Praksis i København, Center for Sundhed og Samfund, Copenhagen University. Oakley A, Strange V, Bonell C, Allen E & Stephenson J (2006) Process evaluation in randomized controlled trials of complex interventions. British Medical Journal 332, 413-416. Poulsen I & Lindhardt T (2013) Forskning - fundamentet for udvikling af sygeplejen. Danish Journal of Nursing 1, 63-65. 10