Use of Objective Structured Clinical Examination in a Senior Baccalaureate Nursing Course for Assessment of End of Program Outcomes Diane Marcyjanik, MSN, EdS, RN Nita Johnson, MSN, RN
Presentation Objectives 1. Apply the use of OSCE in an undergraduate curriculum. 2. Identify the potential use of OSCE for end of program outcome assessment.
Definition of OSCE An observed examination of clinical skills in a controlled simulated environment, with the utilization of structured checklists, simulators and standardized patients. (Jones, Pegram & Fordham, 2010; Meechan, Jones, & Valler, 2011; Paul, 2010)
Basic Principles Objective structured assessment of clinical skills A series of stations Predefined competencies Use of simulators or standardized patients Students move between stations at timed intervals
Role of the Examiner OSCE examiner (faculty) is an observer. Examiner does not ask questions Only observe performance behaviors and mark the standardized checklist accordingly Some stations may require students to answer written questions without observer present
Why OSCE? Healthcare industry acknowledged students not ready for clinical practice (Kardong-Edgren, 2013) Highly adaptable to a variety of competencies (Bartfay, 2004) Needed objective method to assess clinical skills prior to graduation Clinical skills not objectively assessed after first semester junior year
SPRING 13 FALL 2013 SPRING 2014 FALL 2014 SPRING 2015 FALL 2015 Proposal to Determine UGCC station number Determine assessment areas types & timing Prepare rubrics & debriefing Pilot 1 held Pilot 2 held Pilot 3 cancelled Debrief participants Debrief participants Begin logistical implementation OSCE added to curriculum Debrief participants Determine related clinical skills Prepare directions for participants Revise stations Decision to cancel Pilot 3 Select models & examiners IRB Approval Begin logistical implementation Select cohort, models & examiners Revise directions & debriefing Develop electronic rubric Revise stations Revise rubrics, debriefing & Pilot 1 Pilot 2 Logistical implementation Revise directions for participants Pilot 3 Final OSCE Select cohort, models & examiners
Pilot Study Purpose: Two pilot studies were conducted to implement OSCE in second semester senior year. Methodology: Qualitative descriptive Framework: Experiential Learning Theory (Kolb,1984) Scenario: Holistic and cultural diversity
Description Orientation Station 1- Knowledge: Ten point quiz on wound healing Station 2- Psychomotor skill: Perform dressing change (mannequin- with voice over- standard set of responses) Station 3- Patient teaching: Homecare instructions for dressing change (standardized patient) Debriefing
Criteria For Passing OSCE Station 1 = 10 % of Total score Station 2 = 70 % of Total score Station 3 = 20 % of Total score Must meet all Critical Behaviors & achieve > 85 % of total points
Implementation of OSCE Rubrics developed for stations Adapted from previous courses Expert review Pilot 1: paper rubrics Pilot 2: electronic rubrics Faculty orientation Student preparation Pilot 1 & 2: list of content areas for OSCE Pilot 2: added orientation power point
Data Collection Student Scores Individual stations and overall score Survey Faculty and students completed online evaluation survey immediately following the OSCE Focus groups Faculty and students participated in discussion around OSCE
Combined Pilot Results (N=17) 16 14 12 10 8 6 4 2 0 Station 1- passed writen exam Station 2- passed perf. skills Station 3- passed pt. teaching Passed OSCE Points 9 6 7 0 Critical Points na 2 12
Survey Results: Students Did you prepare for the OSCE? N = 16 YES NO A LITTLE 3 8 5 How did you feel before starting the OSCE? N = 16 NERVOUS CALM & CONFIDENT 12 4
Survey Results: Students How do you feel now that the OSCE is over? N = 16 BETTER WORSE NERVOUS SAME 11 3 1 1 Do you feel that an OSCE would be a beneficial method of testing skills competency in the undergraduate nursing program? N = 16 YES MIXED FEELINGS 13 3
Survey Results: Faculty OSCE Support I am in full support of adding an OSCE to every semester of the nursing program. I approve of beginning with an end-ofprogram OSCE and moving down through Senior 1 and Junior 2 semesters. identify the best course in which to situate the OSCE each semester.
Survey Results: Faculty Faculty Concerns regarding OSCE I'm not sure we have the resources to give students opportunity to re-take OSCE's until they pass them. Electronic OSCE Rubrics Easy to use Liked ability to write comments Reset automatically
Recommendations OSCE in week 1-2 of second semester senior year Conduct OSCE over 1-2 days (maximum) Student orientation Faculty orientation Use of electronic directions, rubrics, and evaluation tools Score 15 % of course grade Remediation plan
Remediation Plan Develop individualized remediation plan with OSCE team Level 1 1 missed Critical Behavior and > 80 % of total points Practice and review specific content Retake station within 1 week Level 2 >1 missed Critical Behavior and/or < 80 % of total points Practice and review entire content Retake entire OSCE within 6 weeks
References Adamson, K. A., & Kardong-Edgren, S. (2012). A method and resources for assessing the reliability of simulation evaluation instruments. Nursing Education Perspectives, 33(5), 334-339. Bartfay, W. J., Rombough, R., Howse, E., Leblanc, R. (2004). The OSCE approach in nursing education. Canadian Nurse, 100 (3), 18-23. Bensfield, L. A.,Olech, M. J., & Horsley, T. L. (2012). Simulation for high-stakes evaluation in nursing. Nurse Educator, 37(2), 71-74. doi: 10.1097/NNE.0b013e3182461b8c Bultas, M. W., Hassler, M., Ercole, P. M., & Rea, G. (2014). Effectiveness of high-fidelity simulation for pediatric staff nurse education. Pediatric Nursing, 40(1), 27-42. Jones, A., Pegram, A., & Fordham-Clark, C. (2010). Developing and examining and objective structured clinical examination. Education Today, 30, 137-141. doi:10.1016/j.nedt.2009.06.014 Nurse Kardong-Edgren, S. (2013). A wake up call with an objective structured clinical examination. Clinical Simulation in Nursing, 9 (1), e3-e4. Retrieved from www.nursingsimulation.org/article/s1876-1399 (12)00356-8 Kirkman, T. R. (2013). High fidelity simulation effectiveness in nursing students' transfer of learning. International Journal of Nursing Education Scholarship, 10(1), 1-6. doi:10.1515/ijnes-2012-0009 Kirton, S., & Kravitz, L. (2011). Objective structured clinical examinations (OSCEs) compared with traditional assessment methods. American Journal of Pharmaceutical Education, 75(6), 1-7. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall. McWilliam, P. L., & Botwinski, C. A. (2012). Identifying strengths and weaknesses in the utilization of objective structured clinical examination (OSCE) in a nursing program. Nursing Education Perspectives, 33(1), 35-39. doi:10.5480/1536-5026-33.1.35 Meechan, R., Jones, H., & Valler-Jones, T. (2011). Students' perspectives on their skills acquisition and confidence. British Journal of Nursing, 20(7), 445-450. Paul, F. (2010). An exploration of student nurses thoughts and experiences of using a video-recording to assess their performance of cardiopulmonary resuscitation (CPR) during a mock objective structured clinical examination (OSCE). Nurse Education in Practice, 10, 285-290. doi:10.1016/j.nepr.2010.01.004
Diane Marcyjanik, MSN, EdS, RN Clinical Assistant Professor marcyjdl@uwec.edu 715-836-4012 Nita Johnson, MSN, RN Skills/Simulation Lab Coordinator, Clinical Instructor johnsnit @uwec.edu 715-836-5714