A systematic review on educational interventions for learning and implementing evidence-based practice in nursing education: The state of evidence

Transcription

1 Article A systematic review on educational interventions for learning and implementing evidence-based practice in nursing education: The state of evidence Nordic Journal of Nursing Research 2016, Vol. 36(1) 3 12! The Author(s) 2015 Reprints and permissions: sagepub.co.uk/journalspermissions.nav DOI: / njn.sagepub.com Hanna-Leena Melender 1, Lea-Riitta Mattila 2 and Arja Häggman-Laitila 3 Abstract The aim of this systematic review was to gather, assess and synthesize evidence on educational interventions promoting learning and implementation of evidence-based practice in nursing education and their outcomes. Literature was sought from the Cochrane, CINAHL and PubMed databases. The data (10 articles) were analysed by applying qualitative content analysis. The analysis of the interventions produced five main categories: Theoretical foundations, Teachers and partnerships, Learning contents, Learning/teaching methods, and Learning tools. Outcomes of the evaluations were divided into three categories. A Web-based EBP learning module supplemented by help sessions, a program including the Self-Directed Learning Process for EBP Basics and a Workshop for Critical Appraisal of Literature, and an Evidence-Based Practice Project as a group work were interventions that had gained excellent outcomes. When supporting nurse students learning of EBP, it is recommended to use a combination of self-directed learning and a consultation by a teacher when needed, different learning tools, as well as partnerships in planning and conducting interventions. Keywords evidence-based nursing, evidence-based practice, learning, nursing education, outcomes of education, systematic review Accepted: 27 May 2015 Introduction Nursing educators and administrators ask for the reformation of traditional curricula and teaching methods to accommodate the shift towards evidence-based practice (EBP) in nursing. 1,2 Nursing students should base their practice on the best possible evidence available. 3 The competencies of evidence-based nursing practice depend on many factors. First, nursing students should acquire a good knowledge base and skills and a positive attitude towards evidence-based nursing. 1 This question is already relevant in nurse teacher education: Salminen, Melender and Leino-Kilpi 4 found that the weakest skills of nurse teacher students were related to teaching decisionmaking and giving nursing students constant encouragement to seek new information. Phillips emphasizes that finding creative and engaging ways to teach research skills and evidence-based practice in undergraduate nursing programs are essential. 5 Second, nursing students have to learn to evaluate nursing practice and ask the right questions for the search of the evidence and development of nursing care. Besides this, they have to be capable of changing the practice according to the best evidence with other health professionals. In this process, they need competent clinical supervisors in an authentic health-care setting. Unfortunately, some studies show that many staff nurses are not familiar with the terms or processes of evidence-based nursing or current research knowledge of patient care. 6,7 Numminen et al. 8 explored the correspondence between nurse educators and managers assessments of novice nurses professional competence. One of the biggest differences concerned the use of evidence-based knowledge: the educators assessed this competency as fairly high, whereas the managers assessed it as lower. The authors suggested, for example, that the educators might have too positive a view of their students readiness to apply research knowledge, or that during nurses socialization process into their new work environment, the use of evidence-based knowledge is not supported enough. 1 Department of Social and Health Care, VAMK University of Applied Sciences, Vaasa, Finland 2 Retired from Metropolia University of Applied Sciences, Helsinki, Finland 3 Department of Nursing Science, University of Eastern Finland, Finland Corresponding author: Hanna-Leena Melender, Department of Social and Health Care, VAMK University of Applied Sciences, Raastuvankatu 31-33, FIN Vaasa, Finland. hanna-leena.melender@vamk.fi

2 4 Nordic Journal of Nursing Research 36(1) Third, educating nursing staff to be evidence-basedoriented forges a stronger partnership between academic programs and health-care settings than previously. According to North-American researchers Alexander et al., 9 in the current healthcare reform climate, the need for vibrant academic-practice partnerships is clear. This also calls for new models of partnership. Partnerships between higher education and the workplace have traditionally been restricted to clinical practice placements and thesis work. 10,11 A systematic review on educational interventions for learning EBP in nursing education has been published in Finland by Melender and Häggman-Laitila. 12 The review included 10 articles on nine educational interventions, published between the years 2000 and The educational interventions were categorized into three groups: information literacy, searching and utilizing research evidence for a clinical problem, and theoretical studies on evidence-based practice. Most of the interventions were at least somewhat effective, even though the review also included some interventions that were mainly ineffective. This systematic review aims to update knowledge on the topic. The review Aim The aim of this systematic review was to gather, assess and synthesize evidence on educational interventions promoting learning and implementation of evidence-based practice in nursing education and their outcomes. The research questions were: 1. What kind of educational interventions have been used in order to promote the learning and implementation of evidence-based practice in nursing education? 2. What outcomes have been achieved by using these interventions? Search methods and outcome Literature was sought from the Cochrane, CINAHL and PubMed databases (1 January 2008 to 26 February 2013). Search terms, results based on them, as well as inclusion and exclusion criteria are described in Figure 1. The titles, abstracts and full texts were screened by two researchers. In order to comply with the criteria, 10 articles were selected for the final analysis (Table 1). Nine interventions focused on nursing students, while in one intervention, the focus was on strengthening a curriculum by educating faculty members. Quality appraisal The quality of the studies was evaluated with criteria presented by Gifford et al. 13 which are design-specific quality assessment criteria for quasi-experimental (8 criteria), survey (6 criteria) and qualitative studies (11 criteria). When a methodological triangulation was involved, the quality of the paper was graded by using criteria meant for the method predominant in the study. Each criterion was given a score from 0 to 2; 0 points representing many limitations, 1 point some limitations and 2 points representing excellent quality of the criterion. The highest possible scores were 16 for quasi-experimental studies, 12 for surveys, and 22 for qualitative studies. Results of this evaluation are presented in Table 1, in association with details on study designs, authors, publication years and countries. Data abstraction and analysis A descriptive matrix of the full texts was drawn up and included information about the authors, publication years, countries of origin, quality grades of the studies, the interventions and participants, methods and evaluation results. The information in the data matrix was analysed by qualitative content analysis. 14 Quantitative and qualitative studies were combined in same analysis because the analysis was descriptive. Information from the data matrix was reduced and the reduced expressions were grouped into subcategories based on their similarities and differences. Only the manifest content was analysed. Abstraction was continued until the subcategories excluded each other. The process was continued to combine subcategories with similar contents to form main categories. The categories were named based on their contents. After categorizing the data, the authors returned to studying the original material, in order to specify the contents and research evidence of the categories. Validity and ethical considerations To ensure a systematic and extensive search process, the search was carried out with a library information specialist and by utilizing database directories. Search terms were chosen to produce a wide range of terms focused on evidence-based nursing, considering the word indexes and special features of the databases. These were reported accurately to ensure repeatability (Figure 1). Two researchers working independently made the selection. However, the selection process and ambiguous cases were discussed together, which added to the reliability of the data. The articles were first chosen based on their titles. Due to this limitation, there is a possibility that some studies may have been left out. On the other hand, the manual search of the reference sections of the articles chosen for the review did not indicate any need for the search process to be repeated. When screening the titles, no possibly eligible publications published in any other language than English were found. Thus, it is unlikely that language or publication bias exist in the review. The review comprised articles from several journals and different cultures including also non-english-speaking countries. Using several databases also decreased the likeliness of bias in acquisition of material for the review. Careful documentation of all relevant information about the original studies to the matrix and careful use

3 Melender et al. 5 Search terms (1/1/ /2/2013) Cinahl (Boolean phrase): (MH "Research, Nursing") OR (MH "Professional Practice, Research-Based") OR (MH "Nursing Practice, Research-Based") OR (MH "Nursing Practice, Evidence-Based") OR (MH "Practice Guidelines"), (MH "Quality Improvement") OR (MH "Quality Assessment") OR (MH "Quality Management, Organizational") OR (MH "Evaluation and Quality Improvement Program") OR (MH "Quality of Nursing Care"), (MH "Collaboration"), Peer Reviewed; Research Article Cochrane: '"Evidence-based nursing", Cochrane Reviews, Other Reviews, Trials, Methods Studies, Technology Assessments, Economic Evaluations, Cochrane Groups PubMed (keyword, Map Term to Subject Heading): "Evidence-Based Nursing" AND ("nursing education" OR collaboration OR quality OR development* OR improvement*), Case Reports; Journal Article; Clinical Trial; Evaluation Studies; Meta-Analysis; Practice Guideline; Randomized Controlled Trial; Review Inclusion criteria published in English abstract available describes implementation of evidence based practice (EBP) describes educational interventions in nurse education Inclusion criteria describes implementation of EBP describes educational interventions in nurse education student nurses an empirical study full text available Exclusion criteria Titles: 1028 Cinahl= 320 Cochrane= 47 PubMed= 661 Abstracts selected on a basis of a title: 72 Cinahl= 12 Cochrane= 2 PubMed= 58 Full texts selected in a basis of an abstract: 38 Cinahl= 3 Cochrane= 1 PubMed= 34 Full texts selected for review: 10 Cinahl= 1 PubMed= 9 describes distribution of research knowledge or search strategies and partnering with library or practicing nurses describes availability of research articles or information literacy describes an educational intervention in nurse education or a development project that has not been evaluated by an empirical study describes an educational intervention for clinical supervisors of student nurses or post-registration nurses and midwifes describes an educational intervention on critical appraisal, cognitive and research skills, problem solving or simulation for student nurses, without any connection with evaluation of EBN duplicate publications papers on contemporary issues and other general type of papers Figure 1. Search strategy. of this information in the analysis increases the reliability of the analysis. The categories contained many observations extracted from the data. Repeatability was achieved in the analysis, and it may be assumed that constructing essential conceptual categories was successful even though the scope of data was limited. 14 The analysed articles have been listed and readers may verify the reported classifications based on them. Instruments developed and tested earlier were used in five studies. In two studies, instruments were developed for the purposes of the present study. Validity was reinforced by using quantitative and qualitative data collection methods in six studies (Table 1). The results of the evaluation may be distorted if the data are analysed only by observing the effective interventions or if literature only provides biased or inadequate data ignoring ineffective interventions. Ethical issues were considered in this review, because some of the papers included had also reported partly unsuccessful implementations as well as inadequate available evaluation data. Results Educational interventions To describe the educational interventions, five main categories were identified: Theoretical foundations,

4 6 Nordic Journal of Nursing Research 36(1) Table 1. Educational interventions, evaluation and outcomes. Study design Authors Year of publication Country Quality Educational interventions and participants Evaluation methods Outcomes: A. Excellent; B. Partly good; C. Poor Quasi experimental Fernandez et al Australia, Excellent EBP-DVD computer laboratory teaching method (28 participants) compared with Standard Distance (36 participants), Computer Laboratory (24 participants) and Didactic Classroom (99 participants) teaching methods. Student assignments that comprised validated instruments, outcome measures: students ability to: 1.develop a precise clinical question; 2.undertake a literature search; 3.identify the level of evidence; 4.critically appraise the published research. learning goals: The EBP- DVD method produced statistically significant higher abilities for developing a precise clinical question compared to the standard distance- and the didactic classroom teaching method and higher abilities for identifying the level of evidence compared to other teaching methods. A: Student achievement of learning goals: Statistically significant differences in perceptions of knowledge, attitudes, beliefs and behaviour after the program. Students satisfaction: Students (97%) were satisfied with learning strategies, but suggested more guidance, opportunities for clinical appliance of evidence and to choose by themselves the topics. learning goals: The experimental group had statistically significant higher post-test EBP Knowledge and EBP Use compared to the control group. No statistically significant differences in Attitudes toward EBP and Future Use of EBP. Zhang et al China Excellent A program on the knowledge, attitudes, beliefs and behaviour of EBP: Phase 1: Self-Directed Learning Process for EBP Basics (4 weeks): Phase 2: Workshop for Critical Appraisal of Literature. 75 students in the teaching hospital from 7 universities and colleges. Knowledge, Attitudes and Behaviours Questionnaire for EBP (The KAB Questionnaire) (30 items with dimensions of knowledge, attitudes, beliefs and behaviour about EBP) and a feedback questionnaire (5 items), pretest n ¼ 75 and post-test n ¼ 74. Kim et al USA Excellent Nursing Leadership/ Management Theory and Clinical Practicum courses with the E-FIT intervention including a 2-hour introductory lesson and EBP group projects. Experimental group 88 students Control group 120 students receiving standard teaching. Critical appraisal subject Students from Wollongong (n ¼ 7) and Hong Kong (n ¼ 30). Knowledge, Attitudes and Behaviours Questionnaire for EBP (The KAB Questionnaire) (26 items with dimensions of EBP knowledge, Attitudes towards EBP, Personal application and use of EBP and Future use of EBP) 208 students at 2 universities completed pretest and/or post-test (response rate 89%) Jones et al Australia Some limitations Questionnaire developed for the present study with 6 questions (self-evaluation). learning goals: Students from both cities showed improvements in knowledge in all areas. Statistical significance was not reported. Students satisfaction: Hong Kong students were satisfied to the facilitator and course content and suggested some improvements in the physical environment and teaching aids. learning goals: The students were better prepared to Melnyk et al A strengthened curriculum in 20 Paediatric Nurse Practitioner -programs Questionnaire (structured and open questions) developed for the present study. (continued)

5 Melender et al. 7 Table 1. Continued Study design Authors Year of publication Country Quality Educational interventions and participants Evaluation methods Outcomes: A. Excellent; B. Partly good; C. Poor USA Some limitations Survey Kruszewski et al USA Some limitations (faculties and graduating students). Evidence-Based Practice, Web-based learning modules supplemented by weekly optional classroom help sessions (two thirds of students attended) and projects in a clinical site. 24 students. Respondents: In 2004, 139 graduates from the participating schools and 51 graduates from 13 non-participating schools In 2007, 116 out of 128 graduates from 19 participating schools (response rate 90.6 %) and 54 out of 65 graduates (response rate 87.7 %) from 12 non-participating schools. A performance scale (values and attitudinal aspects of EBP competency) at the end of the course A written course evaluation by students Qualitative focus group interviews for instructors (sample size: unknown). assess and manage children health problems using an evidence-based approach. Reports of screening for certain problems were higher in graduating students from the participating schools. Students satisfaction: In 2007, students in nonparticipating schools were more satisfied with how they were prepared for interventions for children compared with students in participating schools. Several suggestions for the improvement of the curriculum and educational tool kit. Faculty satisfaction: Faculty of participating programs increased their own knowledge. Weaknesses of the programs in participating and non-participating schools were identified in 5 areas. A: Student achievement of learning goals: Highest ability to develop a clinical problem statement, retrieve, critique and synthesize evidence and write an EBP protocol. High value for EBP. Lower ability to implement a practice change. Some students perceived they did not have a sufficient background in statistics. Students satisfaction: High satisfaction with the clinical project. Faculty satisfaction: High satisfaction with the collaborative teaching experience and the evidence received for the improvement of nursing care. Instructors felt more connected to students clinical learning and were very satisfied with students learning outcomes based on poster presentations and projects. B: Students satisfaction: The majority was satisfied with the unit, but the results were not Halcomb & Peters Two one semester units focused on research methods. A questionnaire utilized across the university to provide student feedback (13 quantitative and 2 (continued)

6 8 Nordic Journal of Nursing Research 36(1) Table 1. Continued Study design Authors Year of publication Country Quality Educational interventions and participants Evaluation methods Outcomes: A. Excellent; B. Partly good; C. Poor Australia Some limitations Mattila et al Finland Excellent Qualitative Heye & Stevens USA Many limitations Lalor et al Ireland Many limitations Nursing journal club. An Evidence-Based Practice Project as a group work. 74 undergraduate nursing students. Library-based classes on electronic database search by librarian and subject lecturer. qualitative questions) to 442 undergraduate nursing students, 369 answered (response rate 83.5%). Previously developed and tested questionnaire (structured and open questions), 53 respondents. Written course evaluation on facts, feelings, attitudes and behaviours, including one quantitative question (returned by 13 groups and 6 individuals, response rate unknown). Students quantitative course grades. Search history files, categorized as either poor, fair or good 108 students from 2008 to At least one within-year comparison was available for 80 students. significantly different from those of other units in the university or school. Few students were dissatisfied. learning goals: The students learnt more often from the research results than were able to utilize them. Younger age, shorter work experience and participation in research and development activities were associated with positive learning outcomes. A: Student achievement of learning goals: Out of 14 groups 12 achieved grades of 90 or above on a scale of 100. Students satisfaction: Students were satisfied with their learning and the group work method. The 6 individual respondents mostly were dissatisfied. The students agreed that the strategy stimulated critical thinking and analyses (mean 4.22 on a scale of 5). learning goals: The sessions in the first and second years of the programs resulted in improvements in the ability to search, with less improvement in third year. Teachers and partnerships, Learning contents, Learning/ teaching methods, and Learning tools. Seven papers included information on the theoretical foundations of the interventions. These were divided into two subcategories: theories related to either learning or models related to evidence-based practice/knowledge. Learning theories included the Constructivist Model of Learning, 15 Self-directed Learning strategies, and Bandura s Self-efficacy Theory. 18 Besides this, research evidence of adult learners was used. 16,19 Models related to evidence-based practice/knowledge included the ACE Star Model of Knowledge Transformation Theory, 20 the four A s of research skills (Awareness, Appreciation, Application, Ability), 21 Rogers Diffusion of Innovation Model, 16,18 and Iowa Model of Evidence-Based Practice. 16 Moreover, the developers of the interventions reported the use of research evidence of the topics related to the subject. 21 The subcategories describing teachers and partnerships of the educational interventions were faculty teachers, clinical teachers, researchers, leaders and a librarian. Faculty teachers were lecturers 16,22 or a facilitator of the topic. 21 In one paper, only faculty 20 was mentioned. Clinical teachers were clinical instructors, 16 clinical preceptors 17 or nursing educators within the health care system. 17 Researchers were either investigators of the study 17,18 or teachers who were active researchers. 19,23 Leaders included the director of the program, 17 a leadership team of five faculties 22 or hospital nurse directors and ward managers. 23 The librarian acted as a teacher in two interventions. 23,24 Information about teachers roles or job descriptions was scarce in some papers.

7 Melender et al. 9 Table 2. Learning contents and learning/teaching methods of the interventions (the numbers refer to the articles). Categories, subcategories, themes Learning contents of the interventions Foundations of EBP - significance of research/ebp X X X X - research approaches and processes X X - basic principles of EBP X X X X X - priorities for clinical improvement X - competences needed by clinicians X to enhance quality and safety Processes of EBP - formulation of a clinical question X X X - basic processes of EBP X X X X X - evidence summaries/systematic reviews/guidelines X X X - evidence search skills X X X X - appraising the quality of research X X X - assessing the level of evidence X X - implementation of EBP into clinical practice X X X X Strengthening a curriculum (learners here: faculty members) - topics related to teaching strategies X - topics to be taught for the students X Learning/teaching methods used in the interventions Classroom learning/teaching of the subject X X X X X X X X Collaborative learning around a clinical topic - selecting a topic X X X X X - acquiring the evidence X X X X - planning an EBP implementation strategy X X X - giving a PowerPoint presentation X X - giving a poster presentation X X X - discussion, the students chairing X Online teaching X X Seminar X Self-directed learning X X X X Individual or group consultation/guidance X X X X X X Assessment - an online test/examination X X - an essay or another written document X X X - an open book multiple choice examination X Facilitation of students learning was also enforced by partnerships, which were described with two subcategories: clinical sites and an external appraisal panel. Clinical sites meant clinical placements where the students conducted their EBP projects in their clinical placements in the guidance of their clinical preceptors 16,18 or where journal clubs were implemented. 23 A panel was convened with representatives from health organizations to assess the format and content of the subject, as part of a Graduate Certificate in Health Research. 21 The learning contents of the interventions were classified into three subcategories: Foundations of EBP, Processes of EBP and Strengthening a curriculum with 14 themes (Table 2). The contents were presented on different levels in the original papers. For example, one paper 22 reported such topics as EBP, which we chose to categorize as part of the theme Basic principles of EBP. Learning/teaching methods of the interventions were classified into seven subcategories (Table 2). Similarly as in the case of the contents, these were also presented on different levels in original papers. Classroom learning/ teaching was the most frequently used method. Collaborative learning around a clinical topic was used second-most usually: two of these interventions were conducted during the faculty learning 17,20 and three in clinical settings. 16,18,23 Many interventions included different learning tools provided for the students or teachers. These were

8 10 Nordic Journal of Nursing Research 36(1) categorized into seven subcategories. One intervention was an EBP-DVD consisting of resources freely available on the World Wide Web to provide students with self-directed opportunities to learn. 15 The students were provided with a website 15,18,19,21 or other study materials or publications. 20,21 For work purposes, the students were provided with templates for appraisal tools 21 or a research critiques worksheet. 17 The teachers received an educational resource tool kit 22 and the students received a learning package, 17 both of which included many kinds of materials. Moreover, the students were provided with a free access card to the university library. 17 Outcomes of the interventions The outcomes of the interventions were divided into three categories: Student achievement of learning goals, Students satisfaction, and Faculty s satisfaction. The first one was prioritized, but in addition to students learning, it is also important to evaluate how the instructional practices work. 25 The outcomes were categorized as follows: A Excellent outcomes (mostly positive outcomes); B Partly good outcomes (some outcomes positive, some not); and C poor outcomes (most of the outcomes poor) (Table 1). Four interventions showed very good outcomes related to student achievement of learning goals. One of them was a project 20 and the others included a Critical Appraisal course, 21 a Web-based EBP learning module supplemented by help sessions, 16 and a program including Self-Directed Learning Process for EBP Basics and a Workshop for Critical Appraisal of Literature. 17 Students satisfaction was high with a Web-based EBP learning module supplemented by help sessions, 16 a program including Self- Directed Learning Process for EBP Basics and a Workshop for Critical Appraisal of Literature 17 and also with an Evidence-Based Practice Project as a group work, in the group evaluations, but not in some sparse individual evaluations. 20 Faculty satisfaction was high when a Webbased EBP learning module supplemented by help sessions 16 was evaluated. Three interventions out of ten were ranked as A, Excellent outcomes 16,17,20 (Table 1). Discussion The first research question of this review dealt with finding out what kind of educational interventions have been used in order to promote the learning and implementation of evidence-based practice in nursing education. The descriptions of the interventions in original papers were diverse and sometimes superficial, and the abstraction of the categories and synthesis of the results in this review remain low. Theoretical foundations were reported in seven papers, and some of them even reported the use of more than one theoretical framework. All interventions that were ranked excellent 16,17,20 included at least one theoretical framework. Learning theories help to plan educational interventions pedagogically 26 and EBP models guide the design and implementation of approaches intended to strengthen evidence-based decision making, 27 and the use of these both can be recommended. Many kinds of teachers and partners had been involved in the interventions. However, it was not possible to see any associations between any group of teachers and the outcomes of the evaluations. What is important for the students motivation and learning is that during their education, they can see examples of partnerships between the faculty and the clinical sites and are exposed to learning tasks in the clinical practice, as was apparent in two partly good interventions 18,23 and in one excellent intervention. 16 Gray 11 states that a partnership has two aims: first, to assist students to experience the clinical reality of evidence-based practice in a non-clinical nursing course and second, to interweave evidence-based practice into the everyday reality of what nurses do. In Gray s study, students identified the research partnership project as the course experience that most positively affected their learning. These examples present new models of partnership emphasized by MacPhee 10 and Gray. 11 Partnerships could also narrow possible gaps between expectations and reality in implementation of evidence-based practice from the perspectives of both faculties and clinical sites. 8 In one intervention, the teaching team consisted of teachers who were predominately active clinical researchers 19 and the students identified the team as one positive aspect of their learning experience. It is likely that teachers who are active researchers bring the element of deep subject knowledge to the teaching of EBP and their functions in the teaching position can be recommended. Learning contents were reported on various levels in different papers. Although categorization was made, no associations were discovered between the contents and the outcomes of the interventions. Learning/teaching methods were partly quite traditional. For example, the workshop method was used only in one intervention. 17 However, this intervention gained excellent outcomes and is thus worth a recommendation. One of the two interventions that included online teaching was rated excellent 16 and one of the four interventions that included self-directed learning was also evaluated as excellent. 17 Moreover, two excellent 16,20 and four partly good interventions included consultations or guidance. It is possible that a combination of self-directed learning and a consultation by a teacher when needed could produce good learning outcomes. Supplying the students with learning tools was reported in five papers. Out of the excellent interventions, one 17 included a large number of different learning tools and this may be one element which has enhanced its effects. As self-directed learning strategies must be taken introduced in an increasing amount in the future, the need for effective learning tools enabling effective learning also increases. The purpose of the second research question was to find out what outcomes have been achieved by using the educational interventions. Three interventions had gained excellent outcomes. These included a Web-based EBP learning module supplemented by help sessions, 16 a

9 Melender et al. 11 program including the Self-Directed Learning Process for EBP Basics and a Workshop for Critical Appraisal of Literature, 17 and an Evidence-Based Practice Project as a group work. 20 However, the crucial elements that made these interventions so successful were not entirely identified. The level of evidence on the outcomes of the interventions remained quite modest. There is a need to conduct more studies on the topic, especially experimental studies that include long enough follow-up periods. In evaluating the outcomes of the interventions, it is important to use valid and tested instruments that are relevant in relation to the purpose of the intervention. Other aspects of EBP that have an impact on nursing students learning have also been studied insufficiently. For example, our knowledge of barriers for evidence-based nursing is strong and studies from different countries repeat the results of the same main barriers. 6 However, there is a scarcity of, for example, knowledge of effective interventions of teacher education and organizational infrastructures of health care regarding evidence-based nursing. 28 As for the limitations concerning the validity of the intervention studies promoting students learning, some of studies had small sample sizes from the point of view of the study design, or the article included no information about sample size. Moreover, in some studies, evaluation, such as students or educators self-assessment, was superficial (Table 1). Some respondents may have chosen to answer in a more socially acceptable way, as compared with those who did not reply. These factors may have skewed the evaluations of the interventions. Only learning results and satisfaction of the students and the faculty were examined as outcomes of the interventions, because the aim of a review is to condense information. Conclusions Based on the review, the following conclusions were made: 1. The level of evidence on the outcomes of the interventions is quite modest and more research will be needed on the outcomes of educational interventions for learning and implementing of EBP in nurse education. 2. When reporting evaluation studies on interventions more exact descriptions of the implemented interventions will be needed. 3. When supporting nurse students in learning and implementation of EBP, a combination of self-directed learning and a consultation of a teacher when needed, can be recommended. Different learning tools can also enhance students learning, and using them is recommended. 4. Using partnerships in planning and conducting EBP study programs is recommended. 5. More experimental studies on the topic are needed. Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Conflict of interest The authors declare that there is no conflict of interest. References 1. Salminen L, Stolt M, Saarikoski M, et al. Future challenges for nursing education A European perspective. Nurs Educ Today 2010; 30(3): Rycroft-Malone J, Wilkinson JE, Burton CR, et al. Implementing health research through academic and clinical partnerships: A realistic evaluation of the Collaborations for Leadership in Applied Research and Care (CLAHRC). Implement Sci 2011; 6(74): Kajander-Unkuri S, Salminen L, Saarikoski M, et al. Competence areas of nursing students in Europe. Nurs Educ Today 2013; 33(6): Salminen L, Melender H-L and Leino-Kilpi H. The competence of student nurse teachers. Int J Nurs Educ Scholarsh 2009; 6(1): Phillips RM. Creative classroom strategies for teaching nursing research. Nurse Educ 2014; 39(4): Carlson C and Plonczynski DJ. Has the BARRIERS Scale changed nursing practice? An integrative review. J Adv Nurs 2008; 63(4): Hagler D, Mays MZ, Stillwell SB, et al. Preparing clinical preceptors to support nursing students in evidence-based practice. J Contin Educ Nurs 2012; 43(11): Numminen O, Laine T, Isoaho H, et al. Do educational outcomes correspond with the requirements of nursing practice: Educators and managers assessments of novice nurses professional competence. Scand J Caring Sci 2014; 28(4): Alexander GK, Canclini SB and Krauset DL. Academicpractice collaboration in nursing education. Service-learning for injury prevention. Nurse Educ 2014; 39(4): MacPhee M. Developing a practice-academic partnership logic model. Nurs Outlook 2009; 57(3): Gray MT. Research odyssey: The evolution of a research partnership between baccalaureate nursing students and practicing nurses. Nurse Educ Today 2010; 30(4): Melender H-L and Ha ggman-laitila A. Na ytto o n perustuvan toiminnan oppiminen hoitotyo n koulutuksessa: Katsaus koulutusinterventioiden vaikuttavuuteen [Learning evidence based practice in nursing education: A systematic review on the effectiveness of educational interviews]. Tutkiva Hoitotyo 2009; 7(4): Gifford W, Davies B, Edwards N, et al. Managerial leadership for nurses use of research evidence: An integrative review of the literature. Worldviews Evid Based Nurs 2007; 4(3): Elo S and Kynga s H. The qualitative content analysis process. J Adv Nurs 2008; 62(1): Fernandez RS, Tran DT, Ramjan L, et al. Comparison of four teaching methods on evidence-based practice skills of postgraduate nursing students. Nurse Educ Today 2014; 34(1): Kruszewski A, Brough E and Killeen MB. Collaborative strategies for teaching evidence-based practice in accelerated second-degree programs. J Nurs Educ 2009; 48(6): Zhang Q, Zeng T, Chen Y, et al. Assisting undergraduate nursing students to learn evidence-based practice through self-directed learning and workshop strategies during clinical practicum. Nurse Educ Today 2012; 32(5):

10 12 Nordic Journal of Nursing Research 36(1) 18. Kim SC, Brown CE, Fields W, et al. Evidence-based practicefocused interactive teaching strategy: A controlled study. J Adv Nurs 2009; 65(6): Halcomb EJ and Peters K. Nursing student feedback on undergraduate research education: Implications for teaching and learning. Contemp Nurse 2009; 33(1): Heye ML and Stevens KR. Using new resources to teach evidence-based practice. J Nurs Educ 2009; 48(6): Jones SC, Crookes PA and Johnson KM. Teaching critical appraisal skills for nursing research. Nurse Educ Pract 2011; 11(5): Melnyk BM, Hawkins-Walsh E, Beauchesne M, et al. Strengthening PNP curricula in mental/behavioral health and evidence-based practice. J Pediatr Health Care 2010; 24(2): Mattila L-R, Rekola L, Koponen L, et al. Journal club intervention in promoting evidence-based nursing: Perceptions of nursing students. Nurse Educ Pract 2013; 13(5): Lalor JG, Clarke M and Sheaf G. An evaluation of the effectiveness of information literacy training for undergraduate midwives to improve their ability to access evidence for practice. Nurse Educ Pract 2012; 12(5): Anderman EM and Dawson H. Learning with motivation. In: RE Mayer and PA Alexander (eds) Handbook of research on learning and instruction. New York: Routledge, Taylor & Francis Group, 2011, pp Illeris K. A comprehensive understanding of human learning. In: K Illeris (ed.) Contemporary theories of learning. London: Routledge, Taylor & Francis Group, 2009, pp Stevens KR. The impact of evidence-based practice in nursing and the next big ideas. Online J Issues Nurs 2013; 18(2): Flodgren G, Roja-Reyes MX, Cole N, et al. Effectiveness of organizational infrastructures to promote evidence-based nursing practice. Cochrane Database of Syst. Rev 2012; 2. Art. No.: CD DOI: / CD pub2.